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ABSTRACT 

This final report discusses the activities and outcomes of 
the Elementary and Middle Schools Technical Assistance Center (EMSTAC), which 
has developed a model of special education technical assistance (TA) to local 
school districts that can be implemented on a national scale. This report 
provides a comprehensive examination of EMSTAC *s support to local school 
districts, reports findings regarding the local implementation of research- 
based practices, and discusses effectiveness of national TA activities. The 
EMSTAC model of TA included three distinct approaches. Strategy I involved 
face-to-face training of Linking Agents (LAs) and financial support for their 
salary and TA activities. Strategy II provided face-to-face training and 
Strategy III provided Web-based training. EMSTAC conducted annual evaluations 
of 32 districts that have completed LA training and found: (1) the delivery 

of TA at the local level takes unique shapes, regardless of the type of 
EMSTAC supports provided; (2) districts in Strategies I and II are positively 
moving through the change process at similar rates; (3) TA is influenced by 
the presence of mediating factors at the local level; and (4) EMSTAC LAs in 
both Strategy I and II have become less dependent upon EMSTAC supports over 
time. Appendices include descriptions of TA activities. (CR) 
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CHAPTER I 
INTRODUCTION 



CHAPTER I 



INTRODUCTION 



The major goal of the Elementary and Middle Schools Technical Assistance Center 
(EMSTAC) has been to develop and evaluate a model of technical assistance (TA) to local school 
districts that can be implemented on a national scale. During the 2001-2002 school year, EMSTAC 
completed its fifth year of a contract with the Office of Special Education Programs (OSEP) to 
design, implement, and evaluate this national model of TA. This deliverable provides a 
comprehensive examination of EMSTAC’s support to local school districts, reports findings 
regarding the local implementation of research-based practices, and discusses the effectiveness of 
national TA activities including recruitment, training of local district-based Linking Agents (LAs), 
continuing TA support, and product development. 

Throughout the report, the names of districts and any identifying information have been 
removed to protect the confidentiality of partnering school districts. Where possible, each district has 
been assigned a number that is consistent throughout the report. 

The EMSTAC model of TA includes three distinct approaches to training and support. The 
approaches, titled Strategies I, II, and III, are distinguished by varying support characteristics, as 
defined in previous EMSTAC reports by Mesmer, Hamilton, & Mclnemey, (1998)' and Hamilton, 
Mesmer, Mclnemey, & Woodmff, (2002).^ Strategy I involved face-to-face training of LAs and 
financial support for their salary and TA activities. Strategy II also involved face-to-face training of 
LAs, but not salary support or financial support for TA activities. Strategy III provided Web-based 
training for LAs, but like Strategy II, this did not include salary support for LAs or financial support 

' Mesmer, E., Hamilton, J., & Mclnemey, M. (1998). The Elementary and Middle Schools Technical Assistance 
Center: Program evaluation proposal. Washington DC: American Institutes for Research. 

^ Hamilton, J., Mesmer, E., Mclnemey, M., & Woodmff, D. (2002). The Elementary Middle Schools Technical 
Assistance Center: EMSTAC Effectiveness Data. Washington DC: American Institutes for Research. 
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for TA activities. Each year, EMSTAC conducted annual, formative evaluations of these approaches 
and their potential for scaling up into larger numbers of school districts. 

The remainder of this chapter includes 1) an explanation of district participation in evaluation 
activities, 2) a review of major annual evaluation findings with a summary of consistent themes, and 
3) an outline of what can be found in the following chapters of this report. 

Participating Districts 

Because of the evolving nature of district participation with EMSTAC and the constantly 
changing number of districts involved, it is important to explain the process used for determining 
which districts to include in the analyses. As a result of our long-term recruitment activities, 
EMSTAC has identified several stages of district participation, defined by the level of commitment 
the district has made to working with EMSTAC. These stages include 1) initial district awareness of 
EMSTAC resources, 2) district interest in accessing EMSTAC resources, 3) commitment to a 
partnership through designation of a local LA, and 4) continued use of EMSTAC resources via 
completion of LA training and implementation of research-based practices. Exhibit I-l below 
illustrates the stages of district of participation, the number of districts involved in each stage, and 
where the reader can find more information about EMSTAC successes and challenges with districts 
in each stage. 

Over the past five years, more than 3000 local school districts across the country have 
received information about EMSTAC’s TA resources. Of these 3000 or more districts, 166 school 
districts expressed an interest in working with us by requesting an EMSTAC Follow-up Packet, 
indicating next steps for participation. As a result of this contact, 69 districts began EMSTAC’s LA 
Training, with various forms of assistance from EMSTAC support staff Of these 69, 58 districts 
have LAs who have completed training, although not along the same timeline. In order to gather 
reliable and valid data regarding technical assistance activities, EMSTAC decided that the criteria for 
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inclusion in data analyses should be at least six months of activity after completion of LA training. 
The deadline for this year was May 1, 2002. Therefore, data for 26 of the 58 districts were not 
included in this year’s analyses for one of two reasons; either it had been less than six months since 
the completion of training by the deadline date, or the district became inactive after training was 
completed. Thus, the analysis for this report focuses on 32 of the 58 districts that have completed 
training. 
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EXHIBIT 1-1: 

Summary of Havelock Stage by District 



Site 


District Havelock Stage 


STRATEGY 1 




District 1 


Extend ^ 


District 2 


Try 


District 3 


Extend 


District 4 


Extend 


District 5 


Renew ^ 


District 6 


Extend 


District 7 


Extend 


STRATEGY II 




District 8 


Try and Extend 


District 9 


Try 


District 1 0 


Try 


District 1 1 


Extend 


District 12 


Extend ^ 


District 13 


Extend 


District 14 


Try 


District 15 


Try 


District 16 


Acquire 


STRATEGY III 




District 17 


Try 


District 1 8 


Acquire and Try 


District 1 9 


Examine 


District 20 


Try and Examine 


District 21 


Relate and Examine 


District 22 


Acquire 


District 23 


Try and Extend 


District 24 


Extend and Examine 


District 25 


Examine and Acquire 


District 26 


Try 


District 27 


Try 


District 28 


Extend 


District 29 


Try 


District 30 


Examine 


District 31 


Try 


District 32 


Acquire 


^ This district is in its third topic. 
^This district is in Its second topic. 


2 This district is in its second topic. 
^This district is in its second topic. 
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EXHIBIT 1-2: 



Stage of Participation 


#of 

LEAS 


Activity 


Relevant EMSTAC Analyses 


Districts aware of 
EMSTAC 


3000 

+ 


Received EMSTAC Info 


• Recruitment and related 
costs 


Districts interested in 
EMSTAC 


166 


Requested EMSTAC Follow- 
up Packet 


• Recruitment and related cost 


Districts trying a 
partnership with EMSTAC 


69 


Started LA Training 


• Recruitment and related 
costs, Training and related 
costs 


Districts continuing a 
partnership with EMSTAC 


58 


Completed EMSTAC training 


• Recruitment and related 

costs, 

• Training and related costs, 

• Support and related costs 


Districts meeting 
evaluation requirements 


32 


Completed training and 
conducted LA activities for six 
months 


• Recruitment and related 

costs, 

• Training and related costs, 

• Support and related costs, 

• Impact Outcomes 



Review of Previous Findings^ 

The last three EMSTAC evaluation reports have progressively presented findings regarding 
EMSTAC’s support to local districts, as implementation of EMSTAC’s three technical assistance 
strategies has unfolded. This section summarizes key findings from the past three evaluation reports, 
and highlights common themes that have emerged across findings. 



^ See the following reports for findings detailed in this review: 

Mesmer, E., Ritter, S., Hamilton, J. & Mclnemey, M. (1999). The Elementary and Middle Schools Technical 
Assistance Center: Evaluation report and adjustments from Year 2. Washington DC: American Institutes for 
Research. 

Mesmer, E., Paulsen, C., Ritter, S., Shami, M., Hamilton, J.„ & Mclnemey, M. (2000). The Elementary and Middle 
Schools Technical Assistance Center: Evaluation report and adjustments from Year 3. Washington DC: American 
Institutes for Research. 

Mesmer, E., Ritter, S., Paulsen, C., Carl, B., Dailey, D., Shami, M., Hamilton, J., Mclnemey, M., & Gerver, M. 
(2001). The Elementary and Middle Schools Technical Assistance Center: Evaluation report and adjustments from 
Year 4. Washington DC: American Institutes for Research. 
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Evaluation Findings for the 1998-1999 School Year 



Results from the first year showed that the change process (implementation of research-based 
practices) was indeed occurring in all of the original districts, albeit through divergent models of 
technical assistance and at various rates. EMSTAC identified three technical assistance models 
emerging at the local level during the first year. These included the 1) Insider-Outsider Linker 
Model, 2) Consultant Linker Model, and the 3) Defacto Linker Model. In the Insider-Outsider Linker 
model, the most critical relationships were between the LA and the TA Liaison, as well as between 
the LA and school staff. The Consultant Linker model was found to be similar to the Insider-Outsider 
model, but with less focus on the LA-TA Liaison relationship, as the LA acted like an independent 
consultant to the schools. In the Defacto Linker model, the LA was actually a member of the school 
staff, which had unique advantages and disadvantages created by the LA’s autonomy and ability to 
make large-scale decisions. 

Importantly, the data from year one showed that, regardless of the linker model in place, the 
EMSTAC TA activities did impact the implementation of research based practices in classrooms 
when three specific factors were present. These variables included: 

1) collaboration (district level awareness of, communication about, and involvement with 
the project and LA activities), 

2) flexibility (ability of district staff and the LA to adjust to changes, work with new ideas, 
and overcome obstacles), and 

3) trust (established interdependence and reliable relationships among district staff, between 
staff and the LA, and between the LA and EMSTAC). 

These findings launched the evolution of a set of “predictor variables” influencing change, with 

shifting operational definitions and refinement each year. In the first year, collaborative relationships 

among district staff, between districts and researchers, and with EMSTAC TA Liaisons, were critical 

to the success of each of the TA models. Flexibility was a significant factor for change in all of the 

local models, as well as a necessity at the national level. Trust was important to establishing and 
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maintaining the critical relationships influencing change. This was also important at both the local 
and national level, as TA Liaisons established credibility with the district LA, and the LA with local 
school staff, giving the project or outside researchers credibility within the district. 

Evaluation Findings for the 1999-2000 School Year 

During the 1999-2000 school year, EMSTAC implemented Strategy II with approximately 
20 school districts nationwide. This cohort included several small, rural districts as well as larger 
urban districts in various regions throughout the United States. Results from the second year of 
evaluation indicated that Strategy I districts were somewhat further along in the change process, 
which was to be expected since they had been working with EMSTAC for an additional year. 

Despite this small difference in progression through the process, the analyses showed there 
were more similarities regarding the implementation of research-based practices across Strategy I 
and new Strategy II sites than differences. The differences were important, however, because they 
highlighted the influence of mediating factors on the change process. From the 25 identified predictor 
variables, significant differences between strategies were found among five variables, including: 

1 ) time (the length of time the district had been working with EMSTAC), 

2) building leadership (the presence of building level administrative leadership, support, and 
cooperation regarding the implementation of LA activities and research based practices), 

3) collaboration (the extent of internal collaboration among the LA, teachers, and 
administrators within the school or district), 

4) support (the content of the contacts between LAs and EMSTAC), and 

5) TA Liaison preparation and role clarity (the level of TA Liaison experience and 
understanding of his/her role, specific tasks, and expectations). 

The precise level of impact that these variables had on the implementation process at each site was 

undetermined. However, EMSTAC determined that three of these variables; building leadership, 

collaboration, and utilization of EMSTAC supports, were significantly correlated with the progress 
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of districts in the change process. Findings also indicated that over time, LAs were beginning to rely 
less on the TA Liaison to move them through the change process. 

Evaluation Findings for the 2000-2001 School Year 

During the fourth year of EMSTAC activity (year three of evaluation activities), EMSTAC 
became fully operational across all three TA Strategies. As of May of 2001, none of the Strategy III 
sites had been involved with EMSTAC long enough to include them in the formal impact analyses 
for the evaluation, but EMSTAC did have enough data from districts in all three strategies to develop 
preliminary analyses on the recruitment process and TA costs. Summaries of these findings are 
below, followed by the 2000-2001 change process results. 

Recruitment Findings 

During the third year of evaluation, EMSTAC was able to report substantive recruitment 
information. Over its four and one-half years of operation, EMSTAC had progressed from 
partnerships with seven initial local districts in three States to working with 58 LEAs in 28 States 
across the country. This growth resulted from the use of multiple recruitment methods, including: 

• “cold” contacts made with local districts; 

• extensive participation in educational conferences and professional development 
meetings; 

• professional referrals from colleagues known by EMSTAC staff; and 

• development of strategic and State-level partnerships. 

In depth study showed that the most effective method of local district recruitment was the use 
of professional referrals, with a 58 percent training completion yield for this recruitment method. The 
second most effective recruitment method was the use of strategic partnerships, with a yield of 33 
percent. The third most effective method of recruitment was participation by EMSTAC staff in 
conferences and professional development meetings. This approach provided the largest number of 
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districts interested, but only eight districts completed training; a yield of 21 percent. The cold contact 
recruiting method initially led to a 27 percent yield, but this yield declined over time, with many 
districts dropping out after the initial EMSTAC contact. Updated recruitment findings are reported in 
Chapter Three. 

Cost Findings 

In the 2001 EMSTAC Report and Adjustments, we also reported cost findings from our work 
with districts. The findings highlighted three functions of cost, including 1) the recruitment of school 
districts and LAs, 2) the training of LAs, and 3) the technical assistance support to LAs and districts. 

Results of the analysis included two key findings. The first finding was that expectations and 
appearances related to cost data do not always match actual expenses and long-term realities. For 
example, it may be sensible to assume that Strategy III training would cost less than Strategies I or II, 
due to support through the World Wide Web. However, Strategy III training was actually more 
expensive to plan, design, and implement and more costly in terms of the on-going support required 
to help LAs complete the on-line training during the first year. (It is expected, however, that this 
strategy will become more cost effective as more districts are involved, thus lowering the cost of the 
strategy per district). 

The second finding was that there is variation within strategies on the functions of cost, as 
well as across them. For example, the analysis detected great variation within Strategy II support 
costs. Some sites required very little to support the district, while others actually surpassed Strategy I 
support costs. Also notable was the observation that it was more costly to support more active LAs, 
due to the number of activities and need for EMSTAC interaction. Therefore, the costs associated 
with support had more to do with the nature of LA activities than with the actual TA strategy 
designation. As with the recruitment information, updated cost findings are reported in Chapter Three 
of this report. 
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Change Process Findings 

Key findings from the impact analyses indicated that regardless of TA strategy, districts were 
moving positively through the change process, and that Strategy I and II sites were now in similar 
stages of the change process, a change from the previous year. Like the year before, results indicated 
that LAs across strategies appeared to be engaged in similar types of support activities. Also, data 
continued to show that LAs relied less and less on EMSTAC supports, but continued to progress 
through the change process independently. LAs reported that the most valuable EMSTAC support 
was the access to research-based information and practices. 

Despite these similarities, LA interview data showed that Strategy I LAs perceived EMSTAC 
supports to be more useful than Strategy II LAs. This was in spite of results indicating that EMSTAC 
supports were factors influencing both strategies (specifically, the access to research-based 
information and the administrative and logistical support of the TA Liaison). One explanation for the 
difference in the reports was the nature of the LA position in Strategy II districts, where most 
technical assistance efforts focused on sustaining interventions or change efforts that began prior to 
EMSTAC involvement. 

Consistent with previous findings, EMSTAC found that regardless of strategy, TA models at 
the local level were divergent. Results from 2001 indicated that each district’s model was unique, 
sometimes linked by the common presence of certain factors or predictor variables, rather than by 
common structures or operational practices, as they were after Year 1. The important variables that 
correlated with district movement through the change process included; 

• visits (the number of EMSTAC site visits to the district), 

• support (specifically, administrative and logistical support from the EMSTAC TA 
Liaison), 

• district leadership (district level administrative support, leadership, and cooperation 
regarding the initiative), 

• past initiatives (the existence of reform initiatives of various kinds in the last five years). 
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• collaboration (the extent of internal collaboration among the LA and key players within 
the district), 

• internal locus of change (specifically, a school or district-initiated effort to address the 
need), and 

• the reason for becoming involved with EMSTAC (if the district became involved through 
a personal or professional connection with EMSTAC). 

The data collected and insight gained fi'om working with the districts during this year led to the 

development of several hypotheses and questions for further study. 



Summary of Previous Findings 

As mentioned previously, this review does not include findings fi'om this year’s evaluation 
activities, as those will be reported in the next three chapters of this report. Nor does it include all the 
findings fi'om our previous evaluation reports but, rather, those that are the most significant and 
common across the last three reports. The following are the most prevalent, general themes that have 
emerged: 



• The delivery of technical assistance at the local level takes unique shapes, regardless 
of the type of EMSTAC supports provided (i.e., type of TA strategy). Therefore, 
technical assistance must be individualized and interpersonal. This result disproves 
the notion that similar supports provided at the national level will result in districts with 
similar local models of technical assistance implementation. It supports the notion that 
technical assistance is locally driven, demanding that a national support entity provide 
individualized and interpersonal support that is meaningful to districts. 

• Regardless of the model of TA that takes shape, the districts in Strategies I and II 
are positively moving through the change process at similar rates. After two years of 
work with Strategy II sites, we found no significant difference in the rate of change 
between strategies. The implementation of research-based practices is occurring in most 
of the Strategy II districts that are served and, therefore, the extra costs associated with 
Strategy I sites do not seem to result in greater progress over time. 

• TA is influenced by the presence of mediating factors at the local level. The fact that 
technical assistance implementation is occurring in both strategies, despite divergent 
district models, may be attributed to the existence of factors that were found to be 
positively correlated with LA Implementation and/or the Stage of Change across years 
and strategies. These are: 
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1) district leadership (Administrative leadership, support, and cooperation at the district 
level), 

2) building leadership (Administrative leadership, support, and cooperation at the 
building level), 

3) collaboration (Extent of internal collaboration between the LA, teachers, and 
administrators within the school or district), and 

4) past initiatives, (Existence of other initiatives related to improving student outcomes 
over the last five years). 

• EMSTAC LAs in both Strategy I and II become less dependent upon EMSTAC 
supports over time. This result indicates that many districts have developed the capacity 
to address local needs using research-based practices. Although the need for support 
diminishes, LAs and involved staff still engage in the implementation of research-based 
practices and view research-based practices as important to improving student outcomes. 

The identification of these themes is critical to EMSTAC’ s work over the last five years, 
because they establish empirically based relationships between EMSTAC supports and the 
implementation of LA activities, as well as between those LA activities and the implementation of 
research-based practices in classrooms in local school districts. They also indicate that it is possible 
for a national technical assistance center to help build capacity at the local level by teaching district 
representatives (LAs) to independently guide fellow educators through the change process. 



Contents of the Report 

In the remainder of this deliverable, EMSTAC will report on the most recent TA activities 
and evaluation findings. Following this introduction, the report is divided into three additional 
chapters and appendices. The content of these chapters is as follows: 

• Chapter II describes the procedures used to conduct this year’s evaluation. In particular, 
we discuss variables of interest, data collection methods, quantitative and qualitative 
approaches used to conduct our analyses, and current limitations to our methods. 

• Chapter III presents the results of our analyses and a preliminary discussion of findings. 
This chapter also presents additional information on the recruitment of districts, 
EMSTAC training and supports, the process of change within each district and how the 
process is affected by a number of factors, the impact of EMSTAC on local school 
district implementation of research-based practices, strategic partnerships for scaling up 
TA networks, and costs associated with each TA strategy. 

• Chapter IV discusses the findings in detail as well as implications for future TA efforts. 
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Similar to previous years, the chapters are followed by extensive appendices. Appendix A 
includes descriptions of local TA activities that took place in school districts this year, and the 
process of change that occurred in those districts. These descriptive summaries are provided for the 
32 EMSTAC districts selected for data analysis: seven Strategy I districts, nine Strategy II districts, 
and 16 Strategy III districts. Again, the identifying information has been removed and each district is 
referred to by an assigned number. Additional appendices include protocols (Appendix B), and a list 
of EMSTAC presentations and products (Appendix C). 
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CHAPTER II 

EVALUATION METHODS AND 

PROCEDURES 



CHAPTER II: 



EVALUATION METHODS AND PROCEDURES 

During the 2001-2002 school year, EMSTAC employed a number of procedures to collect 
evaluation data from seven active school districts participating in Strategy I, nine active districts in 
Strategy II, and 16 active districts in Strategy III. EMSTAC developed evaluation procedures and 
instruments intended to measure predictor and outcome variables at the school and district levels. We 
used multiple methods that offered access to several data sources. This variety of methods and 
sources facilitated the development of a holistic understanding of the impact of EMSTAC’s TA 
strategies by providing access to the experiences and insights of the diverse group of individuals 
involved in the TA process. Once data from the 32 sites were analyzed and integrated as a whole, 
several important influences and outcomes regarding the delivery of our TA strategies became 
apparent. 

This chapter begins with an explanation of how districts and schools were chosen for data 
collection purposes. In the sections that follow, we discuss the variables on which data were 
collected, our reasons for focusing on particular variables and not on others, the procedures for 
conducting site visits and collecting data, the methods for analyzing the collected data, and the 
limitations of the data and analyses. 

Data Used in the Evaluation 

This year EMSTAC conducted its final data collection to evaluate the implementation and 
impact of TA activities in the local sites. Data collection was guided by a set of variables defined in 
the conceptual framework. The conceptual framework mapped out the rationale of how TA provided 
through the EMSTAC model was implemented and sustained with results. The framework comprised 
two sets of variables: 
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• Predictor Variables: These variables might have influenced both implementation and 
impact and, therefore, served as predictors of variation in implementation and impact. 

• Outcome Variables : These variables were an indication of program implementation. 
Implementation was assessed using an implementation continuum scale and other 
indicators of impact. 

Predictor Variables 

Predictor variables guiding data collection are described below. They consist of factors 
related to the local districts and the EMSTAC program. 

District Factors 

Leadership: Administrative leadership, support, and cooperation at the building and 
district levels. The impact of school and district leaders on program implementation is well 
established in the literature (Leithwood et al., 1999/. This impact occurs as leaders communicate 
goals and expectations and coordinate and structure opportunities for building school and staff 
capacity to achieve those expectations. As school principals and other leaders built the goals of 
EMSTAC into their expectations, staff were more likely to receive a consistent set of signals on the 
value of skills and knowledge that they could attain through this TA system. The study 
operationalized these issues in terms of the degree to which administrators worked with the LA and 
knew about the project. 

Locus of change for addressing local needs. This variable focused on identifying where the 
desire to address the need originated. From previous data and experience working with EMSTAC 
school districts, we have developed a sense that commitment to addressing a need might be 
influenced in part by the extent to which the original need was identified externally or internally. 



* Leithwood K., Jantzi, D., & Steinbach, R. (1999). Changing Leadership for Changing Times. Philadelphia, PA: 
Open University Press. 
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Reason for addressing a need. This variable looked at reasons behind a district’s decision to 
work with EMSTAC. Of particular interest was the district’s initial perception of possible benefits in 
addressing particular needs. 

TA needs. This measure looked directly at the needs for which TA services were provided in 
each district and the influence that this identification of needs had on TA implementation and impact. 
TA needs were delineated as follows: 

• Reading 

• Discipline 

• Writing 

• Inclusion 

• Curriculum 

• Disproportionality 

Districts that varied in relation to these particular needs might have had different experiences 
in implementing interventions. 

School and district openness to outside TA. Schools and districts might have varied in the 
degree to which they were open and receptive to receiving outside TA. Variation on this variable 
could have influenced LA implementation and the level of change. This factor was operationalized 
using a scale of 1-5, where one equals not at all open, and five equals very open. 

Existence of other current or past initiatives. Another major factor potentially influencing 
program implementation was the degree to which schools and teachers were seeking to implement a 
wide range of initiatives and whether these initiatives diffused attention and resources or worked 
together to improve instruction and school programs. This factor was operationalized for EMSTAC 
in three levels; 

• No other initiatives in special education were being implemented. 

• Other special education initiatives existed, but they were small-scale or minor. 
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• Other special education initiatives existed, and they were major, large-scale initiatives. 
Data on special education initiatives were collected separately for both current initiatives and those 
with which the school district has worked the past five years. 

Existence of financial resources other than EMSTAC (if applicable) to support the 
initiative. The extent to which districts and schools used resources to assist program implementation 
(in addition to direct support from EMSTAC) was a significant indicator of local support for the 
program. It also served as an additional source through which the capacity to implement EMSTAC 
programs was developed. This factor was operationalized in terms of the degree to which additional 
internal or external resources were devoted to support the initiative. 

Extent of internal collaboration between the LA and teachers and administrators within 
the school or district. Collaboration involved joint activity through which professionals shared, 
learned, and delivered services together as a team. This concept was central to the effective 
implementation of a number of initiatives. This factor was operationalized in terms of the frequency 
with which the LA collaborated with others versus working independently. 

Level of support from external agents other than EMSTAC-groups that have no affiliation 
with EMSTAC. External stakeholder support for program implementation might have also been a 
critical factor in implementing and sustaining a program. Though financial support was an indicator, 
this measure looked beyond financial commitment to explore whether external groups supported the 
goals of EMSTAC services as appropriate aims. 

Student enrollment and demographics. The evaluation examined variation in 
implementation and impact in terms of four different measures of student enrollment and 
demographics: total enrollment, percent students with disabilities, percent minority, and percent 
qualifying for free or reduced-price lunch. Total enrollment was used as a measure of district size and 
was operationalized in terms of absolute number of students enrolled, whereas the other variables 
were measured in terms of actual percent. 
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EMSTAC Factors 



Program status: Was the intervention identified for use prior to EMSTAC involvement? 
Several studies have found that program implementation can be significantly influenced by school 
and district history, in particular the degree to which they have identified the need for the new 
intervention or have begun experimenting and implementing components similar to the new program 
prior to formal program adoption (RAND, 1995). This history creates a state of “readiness” among 
local administrators and staff. This factor was operationalized for EMSTAC in terms of whether or 
not program assistance was identified as a need prior to EMSTAC involvement. 

Type and amount of EMSTAC support. Data were collected separately on three interrelated 
variables that examined issues related to communication between the TA Liaison and the LA: 
average number of times a month the TA Liaison and LA communicated; the number of site visits 
made to a district; and the content of communication between the TA Liaison and LA. The first 
variable was measured in terms of the mean number of contacts per month; the second was the actual 
number of visits. The third variable, the nature of TA Liaison and LA contact, was operationalized as 
one of the following: 

• Request for information/response to request/research 

• Problem identification 

• Administrative issues 

• Update from district 

• Evaluation issues 

• Other 

The nature of communication between LAs and TA Liaisons was fundamental to understanding local 
needs and how interventions were identified and implemented to address those needs. 
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Implementation Outcome Variables 



Implementation was assessed through scores on an implementation continuum scale. This 
measure assessed program implementation on the basis of a wide range of LA activities, as listed 
below in Exhibit II- 1 : 



EXHIBIT IM: 

Implementation Continuum Scale 

Sum of scores (1-3) on the following items: 

□ A needs assessment has been completed (which identified topic district is addressing). 

□ TA needs have been identified. 

□ Goals have been established. 

□ The LA coordinated activities that specifically address the identified area of need. 

□ School or district staffs understand and support LA’s efforts to promote school improvement 
initiatives. 

□ Teachers and administrators seek out the LA for assistance. 

□ The LA provides research-based information to teachers or administrators. 

□ The LA engages in a variety of activities to support school staff in their implementation of 
research-based practices. 

□ The LA activities conducted by the LA are conducted on a wide scale (i.e., several 
classrooms, or several schools). 

□ An evaluation plan is in place. 

□ The LA documents his or her TA activities. 

□ The LA communicates with the TA liaison. 

□ The LA conducts his or her own searches for information or research. 

□ The LA has attempted to spread the research-based practice beyond the initial environment. 

□ The LA has supported the district’s capacity to sustain the use of a particular program or 
intervention. 

□ The LA has engaged in renewal activities. 

The scale identified a number of issues central to understanding the implementation of 
change initiatives in the local sites, such as conducting a needs assessment, establishing program 
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goals, and having the LA provide a variety of supports to facilitate school or district use of research- 
based practices. The scale also moved beyond implementation and was used to begin collecting data 
on efforts to sustain the program and to identify a new TA need. 

In the past, a second outcome variable was described, relating to Havelock’s model of 
change, which has served as the theoretical rationale undergirding EMSTAC. According to this 
model, districts receiving TA through EMSTAC would proceed through seven stages of change 
relating directly to implementation, sustainability, and scaling up best practices. The evaluations have 
collected data measuring the stage of change for each district receiving EMSTAC TA services. These 
stages are associated with the following terms: 

1 = Care 

2 = Relate 

3 = Examine 

4 = Acquire 

5 =Try 

6 = Extend 

7 = Renew 

The stage of program implementation based on Havelock’s model of change has served as an 
assessment of both implementation and impact. 

Although used as an outcome variable in past evaluations, Havelock’s model of change was 
not used this year as an index of measuring a district’s progress for the purpose of analysis. The 
reason for this change is due to a “ceiling effect” embedded in the structure of the model. That is, the 
Havelock model was not used for analytical purposes because the scale measuring progress (the y- 
scale) was limited and did not take into account a condition where a school district’s progress 
extended beyond 7 (Renew), the maximum point of the scale. An example best illustrates this ceiling 
effect: School District A has moved beyond stage 7 (Renew) and has begun addressing another area 
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of need. The District would begin back at stage 1 (Care) and would appear to have moved backward 
rather than forward, according to the scale measuring the school district’s progress. In another 
district, the LA has proceeded through all of the Havelock stages but has not scaled-up to other 
schools and has not identified, yet, another TA need. This district will appear as one that has 
“plateaued,” but will still be judged to be further along than the district in the first example. The 
report will indicate where each district is relative to the model of change; however, this information 
will be reported in a descriptive fashion. It should also be noted that “impact sum,” reported in last 
year’s evaluation for analytical purposes, will also be reported for descriptive purposes. 

Impact Variables 

Data were collected and analyzed on two types of impact variables: EMSTAC’s impact on 
LAs, and its impact on child outcomes. The impact variables focused on the LA were designed to 
measure the degree to which EMSTAC contributed to developing the capacity of the LA to promote 
school improvement. This included assessment of the LA’s understanding of the change process, 
capacity of the LA to support implementation of research-based practices, and the ability of the LA 
to scale up the use of research-based practices (e.g., fi'om one classroom to many classrooms or the 
entire school building or district). 

EMSTAC was designed primarily to provide TA that improved local capacity to help schools 
and teachers use practices that have been validated as improving child outcomes. Therefore, the 
EMSTAC evaluation focused on assessing TA delivery and local capacity building in transferring 
research into practice, and not on student outcomes related to these research-validated practices, 
which have already been measured through other well-designed projects. Still, this year, EMSTAC 
collected examples of child outcomes obtained fi'om a representative sample of 14 EMSTAC school 
districts. These particular outcome variables were collected and organized across four categories: 1) 
behavior/school-wide discipline, 2) reading and other academic instruction, 3) inclusion/accessing 
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the general education curriculum, and 4) disproportionate representation of minority students in 
special education. Some caution should be used in generalizing or drawing conclusions from these 
particular findings because they were not collected from a controlled experimental design. 
Nevertheless, they do provide a basis for understanding the logical chain of how EMSTAC TA 
delivery has ultimately connected with improved student outcomes. 

Data Collection 

This section provides a review of the procedures used to organize site visits, a description of 
the data collectors during site visits, and the various methods used to collect evaluation data. 

Selection of Data Collection Sites 

Across the five years of the project, EMSTAC has provided TA to 58 districts: eight Strategy 
I districts, 19 Strategy II districts, and 31 Strategy III districts. All Strategy I districts were chosen to 
participate in the evaluation. They represented a baseline for following changes over time (since the 
Year One evaluation). Of the total Strategy II districts, nine were included in this year’s evaluation. 
Finally, 16 Strategy III districts were included in this evaluation. Only districts receiving LA training 
at least six school months prior to the data collection date have been included in the evaluation. The 
six-month rule was designed to ensure that districts had a fair amount of time to initiate their TA 
efforts before evaluating the effect of EMSTAC’s TA provision to these sites. In this manner, school 
districts would not be unfairly evaluated if their training happened to occur close to the evaluation 
date. Districts trained in the summer and fall participated in the spring wave of the evaluation each 
year. Districts trained in the winter and spring participated in the fall wave of the evaluation each 
year. The evaluation process attempted to capture each district at similar time intervals of 8, 20, 32, 
and 40 months after it began to work with EMSTAC (+/-1 to 2 months). For most Strategy III 
districts, we obtained data on only the first of these intervals (eight months) prior to the conclusion of 
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this contract. As such, 14 districts recently trained were not included in the analysis. An additional 12 
districts that received training are no longer engaged in ongoing collaboration with EMSTAC. These 
districts are also not included in the analysis. 

Within each district, our evaluation focused on every school that was implementing the 
district’s chosen program. In some cases, we did not visit every school in a district for various 
reasons, including time and resource constraints. In those cases, we collected data only in the schools 
that represented the district in terms of the kinds of interventions and activities that the LA was 
supporting. 

Procedures for Site Visits 

Once districts were chosen for participation in the spring wave of the evaluation, TA Liaisons 
contacted the district’s LA(s). The LAs helped EMSTAC staff identify schools and contacts within 
those schools. The LAs were also extremely helphil in arranging interviews and, in some cases, did 
all the scheduling for our data collectors. 

Data Collectors 

The data collectors were EMSTAC staff members, typically TA Liaisons. In most cases, data 
collectors were assigned only to districts in which they had no previous involvement. Generally, two 
data collectors visited each site. In addition, data were collected from Strategy I and II sites through 
telephone interviews to conserve resources and to acknowledge that the districts had already been 
visited at least twice previously to collect evaluation data. 

Methods for Collecting Data 

Data collection methods included (1) interviews and (2) records-based data collection. It 
should be noted that the use of data collection instruments varied from site to site based on differing 
district level activities and stage of change. 
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LA Interview 



The LA interviews were conducted as part of the site visits (or telephone interviews). The LA 
Interview (see Appendix B) was designed to gather in-depth information about LA roles and 
responsibilities. One of three forms of the LA interview was used, depending on how long the district 
had been working with EMSTAC. Additionally, interviewers asked questions about TA-related 
activities and programs and their effectiveness in achieving program goals. A number of other issues 
were addressed, including challenges and barriers faced, the extent to which teachers implemented 
program components, school and district contextual information, and feedback on EMSTAC’s level 
of support to the LA. 

“Other” Interview Protocol 

Whenever possible, EMSTAC site visitors met with other school or community members to 
learn more about a district’s program(s) and level of implementation. Persons interviewed included 
building and district administrators, school counselors, behavioral interventionists, reading 
specialists, and district specialists. The “Other” Interview (see Appendix B) was designed to gather 
general information about their understanding and involvement with the program and their level of 
satisfaction. This interview was also designed to capture important contextual information about each 
district and school. 

TA Liaison Survey 

EMSTAC surveyed each TA Liaison on the EMSTAC staff. During the interview (see 
Appendix B), the TA Liaisons were asked to describe the districts they were responsible for and the 
extent of their communication with the districts, as well as their perception of how well EMSTAC 
served each district’s TA needs. 
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Document Collection 



Several documents were reviewed from each district or from internal EMSTAC records to 
gather information on contextual variables, the extent of communication between EMSTAC and 
districts, and the nature of that communication. These documents included School District 
Information Forms, Organizational Assessments, and TA Communication Logs. The EMSTAC 
evaluation team collaborated with TA Liaisons to ensure that their communication with districts was 
reflected in these documents. 

Procedures for Analyzing and Synthesizing Data 

This report contains both qualitative (descriptive) and quantitative summaries of the data. 
The methods used to analyze the qualitative and quantitative data, and to develop appropriate 
synthesis of findings, are discussed below. 

Cross-Site Qualitative Analysis 

The goal of the qualitative analysis was to make sense of the voluminous data collected in 
ways that capitalize on continuing refinement and ensuring maximum understanding of the concepts 
and relationships being studied. Essentially, qualitative analyses for this evaluation were designed to 
corroborate and explain findings from the quantitative analysis, identify new leads, and provide rich 
examples of EMSTAC implementation and impact from local districts and schools where TA 
services and schooling came to life. The descriptive summaries for each district were used as the 
basis for a more overarching cross-site analysis related to key variables in the conceptual framework 
found across the study sites. The analyses were conducted using a progressive building-block 
approach that began with site visits, built on these data to develop descriptive summaries, and 
eventually moved from descriptive summaries to cross-site analysis. The following procedures were 
utilized: 
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• Developed predetermined first-level codes (described below) that were based on the 
conceptual framework to guide data collection, 

• Developed preliminary summaries for each contact after each site visit as a way to think 
about how data might or might not correspond to existing codes and to identify emerging 
issues and leads that should be brought into future data collection, 

• Used the first-level codes as a guide to review transcripts and other sources of data (i.e., 
descriptive summaries, sorting data into files), 

• Used data analysis to develop and expand upon pattern codes that began to connect 
variables, 

• Developed the descriptive summaries, 

• Used the first-level codes to identify patterns cutting across local sites, and 

• Applied propositions that connected patterns into relationships and larger themes. 

Data Organization and Analysis During Fieldwork 

Analysis first began during data collection as a way of cycling between assessing existing 
data and generating strategies for collecting new data. This process more directly linked the analysis 
of an ongoing, interactive enterprise to what emerged in the field. Thus, data collection and analysis 
were interwoven from the beginning. Field visits were combined with time for data reduction, 
display, and conclusions. This was facilitated by asking data collectors to summarize their 
impressions and findings related to key variables under each interview question. 

First-Level Codes 

Qualitative research was primarily conducted with words, usually in the form of written-up 
field notes and documents that contain words. Words accumulated quickly because every detail 
might have appeared important in the beginning, and the researcher would have experienced word 
overload. This phenomenon was addressed by developing a “start list” of descriptive codes from the 
conceptual framework prior to fieldwork. 
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Data Storage and Retrieval 



Data collected from site visits were placed in manual and computer files organized by key 
dimensions, such as people, settings, and variables at each school. These files enhanced the search 
for patterns by organizing the analysis around the study’s central questions as manifested in the 
conceptual framework. The files were used in combination with multiple data sources (when 
available) to support findings. 

Writing Descriptive Summaries 

Analysis of interviews and data collected from the site visits formed the basis for descriptive 
summaries that were developed for each district. In turn, the summaries served as the foundation for 
cross-site analysis. 

Reviewing Descriptive Summaries 

The descriptive summaries were screened using the revised first-level coding scheme to 
identify and cluster segments of information related to key concepts and themes of the study. This 
process began by reading through the summaries, highlighting key words, sentences, and paragraphs 
that would come to serve as the key units for more detailed coding. Notes, ideas, and questions were 
written in the columns. Each unit of data was compared and contrasted and assigned to a code. Once 
assigned, units were sorted into groups corresponding to the study’s key questions and concepts. 
Through a series of propositions applied to the data assigned to first-level codes, patterns emerged 
that connected variables and began to identify larger relationships and themes. 

Data from these descriptive summaries were synthesized in relevant chapters of this report. 
Data collectors and TA Liaisons worked cooperatively to produce the descriptive summaries for each 
district, which are intended to provide rich information about the environments at each site. 
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particularly surrounding the TA activities of the LA. In order to preserve confidentiality we have not 
included these summaries in this report. 

Quantitative Analysis 

The qualitative, descriptive data were analyzed by a team of trained analysts, who then coded 
the data numerically in a manner that was suitable for quantitative (non-parametric) analyses. 
Specifically, when looking for potential relationships between two discrete variables (variables that 
were coded with two or more categories, such as a strategy that has three categories), we performed 
chi square analyses. When looking for potential relationships between discrete, independent variables 
and continuous, dependent variables (for example, the relationship between strategy and LA scale 
scores), we conducted Wilcoxon Rank Sum tests. The Rank Sum test is a non-parametric equivalent 
of the common analysis of variance (or ANOVA) procedure. Owing to our small sample sizes, the 
non-parametric test was more appropriate than parametric procedures. 

The relationship between two continuous variables such as percent special education and LA 
implementation scores was determined through correlational analyses. Paired sample t-tests were 
conducted in cases where the change in variables over time was explored. 

Limitations of the Qualitative and Quantitative Data 

The evaluation has limitations related to uneven data collection and small sample size. These 
limitations impaired our ability to make comparisons in some instances and to generalize from our 
data to larger populations. These issues are discussed in greater detail below. 

One problem was the degree to which some local sites did not function as clear models of 
their particular strategy. Generally, clear differences were apparent among the three strategies as 
operationalized, though in some cases inconsistencies emerged because of special situations and local 
circumstances. As this occurred, distinctions between the different strategies were not clear-cut. The 
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realities of life in schools and districts made this unavoidable, although it weakened the capacity of 
the evaluation to compare the different strategies. 

TA Liaisons were responsible for record keeping through their communication logs. Because 
of the diversity of Liaisons with varying levels of experience and expertise, data collected from these 
logs were not always consistent. We addressed this problem by building relevant questions into 
interviews to address limitations on particular issues in specific sites. 

In a related vein, the cross-site qualitative analysis was developed from the descriptive 
summaries written by the TA Liaisons. The degree to which the summaries were developed using 
common, systematic methods for coding and analysis was uneven. Therefore, the database from 
which qualitative and quantitative analyses could be conducted was not necessarily of absolute 
strength or consistency. 

We conducted a series of quantitative analyses to add some breadth to the qualitative 
discussions in this report. For the reasons discussed next, these analyses were intended to be 
exploratory and heuristic, to help us better understand the qualitative findings. They are not intended 
to stand alone, nor should they be used as the only measure of the progress of a particular district or 
group of districts. 

An important issue with regard to the quantitative data is that some of the sample sizes are 
small. Although we have increased the number of districts to which we have provided TA by each 
year, the sample sizes were still small from a statistical perspective. 

These issues were addressed by conducting (mostly) non-parametric analyses (e.g., Kruskal- 
Wallis and Wilcoxon Rank Sums Tests) as opposed to the more robust analysis of variance 
(ANOVA) or t-test. Also, since these analyses are exploratory in nature, we have used the decision 
rule (p < .10) as opposed to the more typical (p < .05) as the threshold for reporting statistical 
significance. Thus, when we reported a finding as “significant,” we meant that there was only a slight 
chance (less than 10 percent) that the specific result was due to chance. 
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RESULTS OF QUALITATIVE AND 
QUANTITATIVE ANALYSES 



INTRODUCTION 



In this chapter, the results of the evaluation are reported in detail. The intention of this 
chapter is to present these results with some level of discussion and interpretation and then expand on 
these impressions within Chapter IV. The current chapter is organized into six parts which, for ease 



in-depth picture of our work as a technical assistance (TA) center. For a closer understanding of what 
has occurred within each individual district, the reader is referred to Appendix A, “Descriptive 
Summaries.” The six sections in this chapter, along with a brief description of the purpose of each 
section is provided below. 

□ Recruitment of Districts - Briefly highlights the process used to recruit districts and presents 
findings of the success rate of each of our recruitment methods. 

□ EMSTAC Supports - This section discusses the various supports EMSTAC has made available 
to each of our school districts. Also included is a detailed discussion of our electronic products 
(website, listserv) and how frequently these resources have been utilized. 

□ Strategy Comparison and Predictor Variables - Data are presented on a number of demographic, 
predictor, and outcome variables that compare school districts in Strategies I, II, and III. The 
relationships between several predictor variables and outcomes are also explored. 

□ EMSTAC Impact on School Districts - This section provides an in-depth analysis of how 
EMSTAC supports are impacting and being utilized by LAs. In addition, this section explores 
how LAs are working to support implementation of research based programs in their districts. 

□ Strategic Partnerships - EMSTAC has worked closely with several organizations to recruit and 
provide support to local school districts. This section discusses our collaboration with other 
organizations and how these partnerships have affected our work. 

□ Cost Comparisons - Finally, in this last section findings are presented regarding the costs 
associated with recruitment of districts, training LAs, and providing on-going technical 
assistance. Where possible, comparisons are made across TA strategies. 



of reading, are separated by dividers. These sections are designed to work as a whole in providing an 
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RECRUITMENT OF DISTRICTS 



Recruitment of Districts 



EMSTAC’s recruitment efforts have expanded and intensified as each new LA training 
strategy has been implemented. EMSTAC has successfully recruited 64 school districts across 28 
states, an increase of 24 district partners and eight states over last year’s recruitment figures. Of the 
64 districts, to date 47 have completed the LA training process. In addition to our district partners 
who have started or completed the training process, there are 98 other school districts located across 
the country that indicated interest in EMSTAC resources. This recruitment process is described 
below. The information reported in this section of the report incorporates data from the 47 district 
partners (seven from Strategy I, 1 5 from Strategy II, and 25 from Strategy III) that have completed 
LA training to date. However, as noted elsewhere in this report, only those districts that are actively 
involved with EMSTAC and have been working with EMSTAC greater than six months are included 
in the analyses that follow in the remainder of the report. 

The recruitment and training of district-based EMSTAC LAs has proceeded over the past 
five years according to three primary strategies. Strategy I, involving face-to-face training, salary 
support for LAs, and ongoing TA resources, was developed and utilized in EMSTAC partnerships 
with seven LEAs. This strategy was developed and implemented during the first two years of 
EMSTAC operation (1997 - 1999). 

Strategy II, implemented in year three of EMSTAC operation (1999 - 2000), has been used 
to establish 15 LEA partnerships. This strategy provides face-to-face training and ongoing TA 
resources, but salary support for LAs is not provided. School district partners typically commit a 
portion of the LA’s regular salary to cover their time spent working with EMSTAC on school 
improvement initiatives. 

Strategy III, developed and implemented in years four and five of EMSTAC operation (2000 
- present), has 25 LEA partners that have completed the web-based LA training. The LAs from 
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these local districts do not receive face-to-face training; instead they utilize comprehensive training 
modules, which have been incorporated into the EMSTAC Website. The EMSTAC Website also 
provides extensive resources for addressing specific interest topics (e.g., reading, behavior, inclusion, 
disproportionality), on-line expert chat events, LA discussion boards, computer-based 
videoconferencing, and other support resources. Recruitment activities conducted under Strategy III 
have also led to a significant pipeline of “prospective districts” that have received information about 
EMSTAC and expressed interest in becoming an EMSTAC partner. 

Recruitment strategies included: 

• Cold contacts: Cold contacts refer to recruitment whereby the school district 
representative has no prior knowledge of EMSTAC and learns about us anonymously. 
Cold contacts include cold calls to school districts, cold mailings (2,039 school districts), 
cold e-mail via special education listservs (reached approximately 960 users), and visits 
to the EMSTAC website. 

• Professional referrals: Professional referrals refer to contacts of EMSTAC staff members 
or mutual contacts between EMSTAC and the school district. 

• Recruiting at professional development events: Recruiting at professional development 
events refers to recruitment that takes place while an EMSTAC staff member is 
presenting at a conference. 

• Strategic partnerships: Strategic partnerships refer to contacts that are made directly 
through our strategic partners or through presentations by EMSTAC staff at state or 
regional meetings of school district administrators that were coordinated by our strategic 
partners. 

While multiple methods were used, they are not all mutually exclusive. A particular school district 
may have been exposed to multiple methods of recruitment. For example, a district may have 
received a cold mailing, attended a state regional meeting where EMSTAC presented through a 
strategic partner, and met EMSTAC staff at a conference. 



The Recruitment Process 
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When a school district expressed an interest in joining EMSTAC (which may have occurred 



following their learning about EMSTAC through any of the above methods), a “Welcome Packet” 
was mailed to the contact person. Upon receiving and reviewing this packet, school districts were 
asked to contact EMSTAC if they remained interested. Once EMSTAC received confirmation of a 
district’s commitment to join EMSTAC and a LA was identified by the district, a “Training 
Supplement Packet” was mailed out to help the LA progress through the on-line LA training. At this 
stage, a TA Liaison or team of liaisons was assigned to the Linking Agent. Necessary follow-up was 
conducted by TA Liaisons via e-mail and telephone to ensure that the LA was supported throughout 
the training process and to encourage them to begin the training process. Upon successful completion 
of the LA training, a “To Assist You” packet was mailed out as a reference guide for the LA to 
support them in their initial efforts. In addition, to ensure timely completion of training, we added 
incentives to the recruitment process, including 1.5 Continuing Education Units through the Council 
for Exceptional Children’s (CEC) Professional Development Program and a $100 gift certificate for 
publications available from CEC. Our use of these incentives have had mixed results. Some Linking 
Agents, especially in rural areas, were excited about the Gift Certificate program and redeemed their 
certificates for classroom materials. They also expressed that these incentives were important to their 
decision to partner with EMSTAC. Conversely, although more Linking Agents took advantage of the 
Gift Certificate Program as opposed to the CEU program, a number of trainees did not redeem their 
certificates. Clearly they were not a motivating factor for these trainees. 

The Four Stages of EMSTAC Adoption 

Exhibit III-l provides an overview of the essential steps that local districts progress through 
during the establishment of a partnership with EMSTAC. Initial AWARENESS (Step I) by the local 
district of EMSTAC resources, INTEREST (Step II) of the district to integrate EMSTAC resources 
into local school improvement activities, the decision by the district to TRY (Step III) an EMSTAC 
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partnership by starting the LA training process, and the decision by the district to CONTINUE (Step 
rV) the partnership by completing EMSTAC training represent the four major steps that EMSTAC 
addresses as part of the LEA recruitment and training process. 



EXHIBIT MM: 

The Four Stages of EMSTAC Adoption 



Stage 


LEAS 


Activity 


I. 


AWARENESS of EMSTAC 


3000+ 


Received EMSTAC Info (includes 
brochure mailings) 


n. 


INTEREST in EMSTAC 


163 


Requested EMSTAC Follow-up Packet 


III. 


TRY Partnership w/EMSTAC 


64 


Started LA training 


IV. 


CONTINUE Partnership w/EMSTAC 


47 


Completed LA training 



As a result of the various recruiting methods employed by EMSTAC over the past five years, 
over 3000 local districts across the country have received information about EMSTAC technical 
assistance resources. From this base of recruited districts, 163 local districts have followed-up with 
requests for additional information (e.g., the EMSTAC “Welcome Packet”). Sixty-four local districts 
have moved to the next step of designating an LA and beginning the EMSTAC training. Finally, 47 
local districts to date have completed the training process and are full EMSTAC partners. 

Effectiveness of Recruitment Methods 

Exhibit III-2 lists each of the recruitment methods used by EMSTAC and the resulting 
number of LEAs that requested information, the number of LAs completing EMSTAC training, and 
the district “yield” as a result of each recruitment method. The most effective method of local 
district recruitment has been the use of professional referrals. Of the 43 local districts expressing 
interest in EMSTAC as a result of this method, 23 have completed LA training. This result provides 
a 53 percent training completion yield for this recruitment method. The second most effective 
recruitment method is the use of strategic partnerships. Of the 49 local districts expressing interest 
due to this recruitment method, 12 completed training for a yield of 24 percent. The third most 
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effective method of recruitment was participation by EMSTAC staff in conferences and meetings. 
This approach provided the largest number of district interest (55), but only eight districts have 
completed training to date; a yield of 15 percent. Finally, the cold contact recmiting method has led 
to 15 interested districts and four districts that completed training. The yield for this method (27 
percent) is considered to be the smallest training completion yield due to the many cold contact 
districts that were ultimately dropped from the EMSTAC database due to incomplete information. 
While cold contacts have proven to be the least effective method of recmitment, they effectively and 
inexpensively exposed EMSTAC to a large number of local districts. 



EXHIBIT MI-2: 

Results from Multiple Recruitment Methods 



Recruitment 

Strategy 


LEAS 

Interested 


LA Training 
Completed 


District 

Yield 


Professional Referrals 


43 


23 


53% 


Strategic Partnerships 


49 


12 


24% 


Conferences & Meetings 


55 


8 


15% 


Cold Contacts 


15 


4 


27% 



Within each TA strategy there does appear to be a difference in the level of success of each 
recmitment method. All of the Strategy I districts (100 percent) were recmited successfully through 
professional referrals. Successful recmitment of Strategy II districts included eight by professional 
referral (53 percent), four by recmitment at professional development events (27 percent), and three 
by cold contacts (20 percent). However, Strategy III districts were mostly recmited successfully 
through strategic partnerships (48 percent). The remaining Strategy III districts were recmited using 
professional referrals (32 percent), recmitment at professional development events (16 percent), and 
cold contacts (4 percent). Exhibit III-3 contains information about the method of recmitment for the 
47 districts categorized by strategy. 
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EXHIBIT MI-3: 
Recruitment Methods 



Recruitment Method 


Across 

Strategies 


Strategy I 


Strategy II 


Strategy III 


Cold Contact 


9% 


N/A 


20% 


4% 


Professional Referral 


49% 


100% 


53% 


32% 


Recmitment at Professional 
Development Event 


17% 


N/A 


27% 


16% 


Strategic Partnership 


25% 


N/A 


N/A 


48% 



According to recruitment data from the 47 school districts that have completed LA training, 
professional referrals appear to be the most effective overall method of recruitment as 49 percent of 
all districts have been successfully recruited this way. However, the effectiveness of each method of 
recmitment varies to some degree, based on whether EMSTAC was recmiting for Strategy I, II, or 
III. The use of strategic partnerships was added as a new method of recruitment for Strategy III. 
This has proven to be the most effective method for this strategy (48 percent). While cold contacts 
have proven to be the least effective method across all three strategies (9 percent), it effectively 
exposed EMSTAC to a large number of school districts. 
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EMSTAC SUPPORTS 



EMSTAC Supports 



EMSTAC staff provided LAs with an array of supports throughout the year. The method 
used for each district was consistent with their preferences and capabilities and included traditional 
kinds of supports such as hard copy products to more technologically advanced supports such as 
computer-based videoconferencing. The primary objective of the supports, regardless of the nature of 
these supports, was to provide LAs with access to timely and quality-driven information to help them 
sustain their work at the local level. The following feedback from an EMSTAC LA illustrates the 
goals of our support. "Hello! I am an Integrated LAR teacher. I was recently asked to help start a 
new department for the "new" Middle School, starting next school year. The department will be 
will target non-fiction literacy and math/science conceptual learning. I was very leery until I 
became involved with EMSTAC. Now I know that I will have the support that I will need to 
validate the changes needed in my district to help all. ” 

The EMSTAC infrastructure and its supporting mechanisms enabled TA Liaisons to use 
varying methods and materials to support LAs. The wide variety of supports available through 
EMSTAC facilitated our ability to meet the informational needs of a larger audience. In this section, 
we will describe the nature and use of the various supports used by EMSTAC. Because the face-to- 
face training methods delivered through Strategies I and II were described in previous evaluation 
reports, this report will focus less on previous training support and instead will expand our discussion 
of our findings regarding the Strategy III on-line distance education training. In addition to online 
training provided to LAs, we will describe additional supports such as the public and restricted- 
access sections of the EMSTAC website including the supports within the website such as real-time 
chat events, the bulletin board, and the EMSTAC electronic discussion forum, LA-Exchange. We 
will also describe the nature of the EMSTAC mailbox communications. The section will conclude 
with a focus on our newest electronic support, computer-based videoconferencing. We will provide a 
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description of the videoconferencing program and will include information about what we learned 
about this technology that relates to its effectiveness as a technical assistance tool. Our efforts to 
refine and improve the nature and quality of supports to continuously ensure their continued 
alignment with the needs of our audiences is prevalent throughout all of our these supports. 

Linking Agent Training 
Strategy I 

An intensive support, Strategy I Training, required the cohort of seven Strategy I LAs to 
convene in Washington, DC to receive a two-day face-to-face training program focusing on the 
Havelock Change Process Model. From this training and in the early years of EMSTAC (1998), we 
learned quickly about the need to adapt and improve practices continuously based on the needs of 
audiences. In this event, LAs felt that an interactive format, as opposed to a lecture type format, 
would be more beneficial to their learning; thus, we modified the training curriculum to coincide 
with these needs. The cohort of Strategy I LAs agreed that the experience enhanced their knowledge 
and skills regarding the change process model. 

Strategy II 

Training for Strategy II LAs was also face-to-face; however, LAs were convened in small 
groups on a regional basis, with approximately six persons in each group. Typically, EMSTAC TA 
Liaisons traveled to the location of the LA cohort and conducted the two-day event at a local hotel, 
college, or within a local school district building, or some other location that was geographically 
accessible for attendees. Videotapes and interactive exercises were incorporated into Strategy II 
Training as a way for attendees to apply their learning. The training sessions enabled participants to 
interact and share experiences with colleagues, who often came from school districts with similar 
characteristics. 
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Overall, a majority of respondents indicated that the strengths of this training experience 
were the opportunities to participate in group activities, the training and support materials provided, 
and the knowledge of the presenters. Attendees indicated that the relationships established during the 
two-day regional training would serve as a support mechanism as they implemented new initiatives 
in their districts. During the three years subsequent to the training, some of the LAs did report 
continuing their relationships with other school district colleagues whom they had met during the 
EMSTAC event. 

Strategy III 

Thirty-one districts, with at least one LA have completed the Strategy III Training that was 
provided through an online, interactive, multimedia curriculum. This does not include EMSTAC 
affiliates (e.g. state officials) who have completed the training for reasons other than supporting their 
role as an EMSTAC Linking Agent. Although the content of the online modules was similar to the 
Strategy I and Strategy II material, this forum used video clips, electronic forms, and discussion 
boards to engage the learner in the curriculum. Hard-copy support materials including the To Assist 
You Guide that contained module highlights, worksheets, and additional resource material 
accompanied the electronic materials. Training participants also received the “EMSTAC Stack of 
Support Cards” that contained step-by-step instructions related to accessing specific components of 
the online training and often referred to the more technologically involved aspects of site. For 
instance, some of the support cards included were The Private Section of the EMSTAC Website 
(which provided directions for getting a username and password). Installing Quick-time and Real- 
player (steps to install software necessary to run audio and video clips), and Participating in the 
Training Threaded Discussion (the online training discussion forum). We included these support 
cards to help trainees to be comfortable in accessing the specific technology-driven functions of the 
site. 
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Trainees completed an electronic training evaluation form immediately following their 
completion of the training. We received 24 evaluation forms from the 3 1 districts that completed the 
training this year. One of our major questions focused on whether LAs believed that they were ready 
or prepared to take on the role of a Linking Agent. In the majority of the responses (67 percent), 
comments such as “Prepared and ready to get others involved in the training to form a team ’’ were 
evidence of the LAs’ sense of preparedness in assuming the role. In other instances, LAs reported 
that the curriculum validated what they already knew about change at the local level. For example, 
one LA indicated, “I have gained more confidence in knowing that the training was a validation of 
some of the knowledge and skill I already had. I can do this correctly. ” However, in other cases (33 
percent), LAs offered no feedback regarding their readiness or reported that they were only fairly 
prepared and offered comments such as, “Fairly prepared, although lam unsure of who my clients 
might be. ” The feedback we received regarding the LAs’ sense of readiness was important, since it 
enabled us to provide support to those persons who expressed confusion regarding their pending role. 
We suspected that with a clearer understanding of the expectations for their position as an EMSTAC 
Linking Agent, these professionals would be more confident in their roles and could focus more fully 
on the substantive nature of their positions. 

Most respondents expressed satisfaction with the content of the training material and 
especially liked the flexibility that they had in completing the modules. Common feedback included 
comments such as “This was the first online training I’ve participated in. Hiked the convenience - 
able to do it at 5:00 a.m. (my favorite work time) " and “I believe the strengths of this training were 
the format and self-regulatory aspect. I found that once I started it was difficult for me to stop. I was 
able to complete the entire training in about two and a half days! I have a husband, two young 
children, am active in my association on the state level and am in graduate school...! am 
impressed! " These statements related to the mix of various reporting formats including text, 
graphics, pictures, and tables, which appeared to help trainees think about what they were learning 
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and provided them with a context for how the material can be applied. One LA expressed, “/ thought 
the mix of media was great. The reading was the right amount. It was easy to maneuver and you 
provided several opportunities for reflection and feedback. The charts and accompanying materials 
were excellent. ” Another LA thought that the case studies were important to highlight a learning 
concept and reflected, “I particularly enjoyed the case study, and it added a real life dynamic to the 
training. ” Others pointed to the online discussion forums as a strength of the curriculum since it 
provided trainees with an opportunity to interact with other professionals who were also going 
through the training, "'I really liked the interaction with others in the discussion and chat sections. 
Knowing that others were experiencing the same things was very helpful. ” 

We also asked for recommendations from the LAs regarding training improvements so that 
we could continuously improve our use of online, interactive formats. Most of the feedback we 
received pertained to the difficulties that LAs had in accessing the audio and video clips and 
explained that their outdated computer equipment at times precluded them from using these 
functions. For instance, one LA reported, “I skipped over much of the audio. I would rather read for 
myself than listen to others ” and another LA commented, "I did not have much success with the 
videos. The two that I was able to see were somewhat helpful by connecting the information to a live 
person. ” Additionally, we received comments related to the inability to download or print out some 
of the documents and received recommendations such as “Having more of the information in a 
downloadable format. Much of the information is not printer friendly. ” We will use these comments 
related to curricular and format features to improve our future online interactive forums. 



Electronic Supports and Products 
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EMSTAC electronic support is provided in part through the EMSTAC website in the form of 
research-based written products and materials (e.g. EMSTAC Extras), real-time chat events, a bulletin 
board, the EMSTAC Listserv, LA-Exchange, and the EMSTAC mailbox. Although the majority of 
these electronic supports were made available to LAs in the summer of 1999, during the past year we 
have added to these supports computer based videoconferencing technologies. Some support 
features, including our research-based products, chat events, and the bulletin board, are only 
accessible through the EMSTAC restricted-access website that requires a username and password to 
access. Although these components, including the EMSTAC Listserv, were made available to LAs 
across the three strategies, the majority of participants in these forums were Strategy III LAs. This 
may have been influenced by our greater emphasis on these distance education and electronic support 
tools during our Strategy III recruitment work and by the larger number of LAs within Strategy III. 
The following sections provide specific information about the purpose and use of these support 
mechanisms. 

Website 

The public component of the website served a variety of purposes, including recruitment and 
resource sharing, and has been in operation since August 1999. The site was a way for external 
audiences to get to know EMSTAC through a detailed description of our mission, objectives, and 
activities. We also included links to staff pictures and biographies as a way to make the relationships 
between our staff and our audiences more personal. The quantity of resources and links available 
through the public website has grown markedly throughout this reporting period, as has the number 
of hits to the site. From the middle of September 2001 until June 2002, representing approximately a 
nine month period (the length of an academic/school year), we had 29,344 visits to the EMSTAC 
website by 9,510 unique visitors. Approximately 1,796 unique visitors progressed on from the public 
website to the restricted-access section of the site. Since a password is required to access the 
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restricted-access section, these visits were made by EMSTAC LAs. A visit is defined as the number 
of times a visitor came to the EMSTAC website. Web tracking software, WebTrends, provides 
EMSTAC personnel with information regarding the demographics and usage patterns of site visitors 
such as the day and time of the week with the most volume. These data allowed us to plan web 
events, such as real-time chat events during times that most users typically access the site. For 
instance, as indicated in the Activity Level Exhibit III-4, Activity Level By Day of the Week, we 
learned that Wednesday afternoons are the most visited time; thus, we have tended to schedule real- 
time forums such as chat events and videoconferences during this time. 

EXHIBIT III-4: 

Activity Level By Day of the Week 




2-Mon 



44^ed 



6-Fri 



1-Sun 



3-Tue 



5-Thu 



7-Sat 



Weekdays 



We have overcome some of the limitations described in last year’s evaluation report 
regarding specific usage patterns of pages within the site, by working with our information 
technology personnel to customize our WebTrends tracking reports. We are now able to identify 
which pages within both the public and restricted-access sections are accessed and downloaded most. 
This feature is another tool that we used to ensure that our products and materials were consistent 
with the informational needs of our audiences. This capacity will be discussed in a forthcoming 
section. 





m-14 

53 



American Institutes for Research 



EMSTAC Extras 



EMSTAC TA Liaisons often compiled materials and resources in response to specific 
technical assistance inquiries from LAs. Since other LAs may have similar questions, we compiled 
these responses and formated the information according to a standard EMSTAC report called an 
EMSTAC Extra. These Extras are available to the general public through the EMSTAC public 
website. During this year we added the following information to our website, including a description 
of the titles of the EMSTAC Extras produced.’ 

Introducing a new section of the EMSTAC website: 

EMSTAC Extras, another type of resource, are written by 
EMSTAC staff in response to questions posed by EMSTAC 
- LAs or EMSTAC partners. They are intended to provide 
practical suggestions to address specific questions. 

Current topics, which can be downloaded in both Microsoft Word and PDF formats, include: 

• Selecting an Intervention or Program: A Guide for Education Personnel 

• The "A-ha" Guide to Maximizing School Resources: Tips and Suggestions 

• Reintegration of Students with Emotional or Behavioral Disorders: Strategies for 
Education Professionals 

• Early Career Awareness and Development for Students with Disabilities in Elementary 
and Middle Schools: An Overview of Early Career and Transition Programs 

• Service Delivery Models for Students at-risk of and with Emotional and Behavioral 
Disabilities: Abbreviated Review and Resources 

• Literacy Instruction and Statewide Assessment Preparation for Students with Reading 
(Learning) Disabilities 

• Selecting a Reading Program 

• A Qualitative Description of Collaborative Teams in Today's Classroom 



EMsneDdn 



Elementary and Middle Schools 
Technical Assistance Center 
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EMSTAC LAs and EMSTAC partners, such as professionals from the Mountain Plains Regional 
Resource Center (MPRRC), also have access to the restricted-access component of the EMSTAC 
website. Through this password-protected section, EMSTAC LAs participate in real-time chat events 
and bulletin board discussions, network with the LA Community through the LA Directory, and 
access research-based products. Last year we had 117 registrants to the EMSTAC website; this year, 
the number has grown to 290 registrants, an increase of 148 percent. The distribution of these new 
registrants is as follows: 

• Eighty LAs - not all of these individuals have completed training at the time of this 
report. 

• Thirty-one affiliates - this includes partners, state representatives, and chat facilitators. 

• EMSTAC staff and miscellaneous guests. 

Research-based Products 

Registrants to the restricted-access component of the EMSTAC website have access to 
comprehensive EMSTAC products that are based on current research findings and models in the 
education field. Although these products have a strong emphasis on research findings, they also 
contain practical, practitioner-fnendly sections that can be easily applied in local school settings. The 
practical application of these materials is evident in the feedback we received from an EMSTAC 
affiliate, “This is an outstanding literacy product. It is a carefully worded overview of the major 
issues that must be addressed in the development of literacy, so that the average layperson can 
understand and connect to the complexity of literacy development. While thoughtfully providing a 
research synthesis, it is written in layperson 's language, with many practical examples to guide 
parents and teachers through general strategies for literacy development. ” 

Throughout this evaluation period, EMSTAC Staff through its topical team structure refined 
existing EMSTAC products such as reading instruction, social skills and behavior, English Language 
instruction, deaf literacy, and developed new products. The feedback we received from LAs 
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regarding their informational needs and our continuous scans of research literature guided our 
selection of topics. A comprehensive internal and external review process by both researchers and 
practitioners ensured that our products were responsive to the needs of educators. 

We developed products during this evaluation period in the following topical areas: 

• Autism, 

• Study Skills, 

• Disproportionate Representation, 

• Accessing the General Education Curriculum, and 

• Literacy Instruction. 

We now have the capacity to track the use of all EMSTAC products through the Web tracking 
software, WebTrends, described in the beginning of this section. As indicated in the accompanying 
table (Exhibit III-5), our EMSTAC Extra related to Service Delivery Models for Students at Risk was 
the most downloaded product on the EMSTAC website, having been downloaded 156 times during 
29 discrete visits. The Early Career Awareness EMSTAC Extra was the second most downloaded 
product, at 121 times. 

Our goal was to develop products, based on available research that contained practical 
information regarding models, strategies, and practices. It was our hope that LAs would use these 
materials in two ways. First, we hoped that LAs would use our support and products to make 
decisions about the implementation and maintenance of school interventions. Secondly, we hoped 
that our LAs would serve in the role of a “disseminator” at the local district level, sharing resources 
and information with a wider network of colleagues. The following note from an EMSTAC LA 
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EXHIBIT III-5: 

Most Commonly Downloaded EMSTAC Products 





FILE 


No. of 
Downloads 


Visits 


1 


Service Delivery Models for Students at-risk of and with 
Emotional and or Behavioral Disabilities: Abbreviated 
Review and Resources EMSTAC Extra 


156 


29 


2 


Early Career Awareness and Development for Students 
with Disabilities in Elementary and Middle Schools: An 
Overview of Early Career and Transition Programs — 
EMSTAC Extra 


121 


26 


3 


The ”A-ha" Guide to Maximizing School Resources: Tips 
and Suggestions - EMSTAC Extra. 


59 


31 


4 


Reintegration of Students with Emotional or Behavioral 
Disorders: Strategies for Education Professionals - 
EMSTAC Extra. 


104 


26 


5 


Selecting an Intervention or Program: A Guide for 
Education Personnel - EMSTAC Extra. 


76 


31 


6 


Understanding the Change Process. Article by Linking 
Agent, Diane Thomas about her experiences with 
EMSTAC 


69 


39 


7 


Selecting an Intervention or Program: A Guide for 
Education Personnel (updated)- EMSTAC Extra. 


66 


33 


8 


The ”A-ha" Guide to Maximizing School Resources: Tips 
and Suggestions (updated)- EMSTAC Extra. 


59 


31 


9 


Training Module - Needs Assessment Survey 2. 


32 


18 


10 


Training Module - Needs Assessment Survey 1. 


31 


21 



highlights the use of EMSTAC supports: “/aw constantly accessing the resources which EMSTAC 
has provided. The progress that we have made would not have been possible without the efforts of 
[TA Liaison], and the EMSTAC staff. lam in constant communication with the [TA Liaison] via 
telephone, e-mail, and hard copy. She always forwards new information and discoveries regarding 
disproportionality, and we are always able to utilize those resources. ” 



Chat Events 

During this reporting year, EMSTAC hosted nine real-time chat events. As indicated Exhibit 
III-6, we were privileged to have some of the leading researchers in our field facilitate these events. 
Professionals such as Dr. Bridgie Ford, Dr. Alfi'edo Artiles, and Dr. Brenda Townsend, provoked 
questions and thought, and shared their expertise regarding an array of important topics such as 
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home-school partnerships, culturally responsive instruction, and culturally responsive classroom 
management. 

Although attendance at these events was not as high as was expected, the small groups 
enabled LA participants to engage in in-depth discussion with these prominent researchers about 
topics specific to their settings. Importantly, the transcripts from these events, including the resources 
and literature recommended by the chat experts, are available to those LAs who could not attend the 
event so that they can always access the information generated through the chat. 



EXHIBIT MI-6: 

EMSTAC Chat Events June 2001 - June 2002 



Date 


Facilitator 


Title 


# Attendees 


mm2 


Dr. Nancy Holodak 


Integrating Mathematic Concepts into Your 
Elementary and Middle School Curriculum 


11 


3/05/02 


Dr. Brenda Townsend & 
Dr. Gwendolyn Webb- 
Johnson 


Culturally Responsive Classroom Management 


10 


mm 2 


Dr. Bridgie Ford 


The Importance of Partnerships Between School 
and Community 


9 


12/13/01 


Dr. Alba Ortiz 


Preventing Inappropriate Referrals 


16 


11/8/01 


Dr. Alfredo Artiles 


Culturally Responsive Instruction 


8 


10/18/01 


Dr. Cheryl Beverly 


University Partnerships: Benefits and Process 


10 


8/30/01 


Mr. Lloyd Mattingly 


Reducing Behavioral Referrals, Suspensions, and 
Expulsions: The Use of Social Skills Training and 
Other Behavioral Interventions in the Classroom 


7 


7/26/01 


Ms. Dianne Tracey 


Using Co-Teaching and Teaming Strategies to 
Facilitate Inclusion for Students with Disabilities 


7 


7/12/01 


Ms. Aeneid Mason & 
Dr. Gayle Nakib 


PAM: Prereferral Assessment Model 


21 
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We were pleased this year with the diminished occurrence of technical problems reported by 
our LAs. This may have occurred because of our use of new software introduced at the end of last 
year. Additionally, our LAs’ enhanced technical knowledge due to more frequent chat events may 
have impacted positively upon their familiarity and comfort with this type of electronic forum and, 
therefore, fewer technical problems were experienced. Although we still heard from a small 
percentage of our audience who indicated that their antiquated computer equipment inhibited their 
ability to access the live event, they were pleased to learn that they could still access the content of 
the event through archived transcripts. 

Bulletin Board 

Within the restricted-access section of the EMSTAC website, LAs were able to access the 
EMSTAC bulletin board. This support tool provided LAs with an opportunity to network, share 
resources, and problem solve. In some cases, the Bulletin Board provided a platform for LAs to 
develop personal relationships with their LA peers. LAs have told us that these relationships often 
continue outside of EMSTAC forums. During this reporting year, there have been 27 posts to the 
bulletin board focusing on topics such as early career development and awareness, non-fiction 
literacy across content areas, the change process, and the inclusion of students with emotional or 
behavioral disabilities in general education settings. The low quantity of information communicated 
through this forum may be partially explained by the availability of other EMSTAC supports that 
also provide opportunities for LAs to build a community, share resources, and problem solve, and the 
support available from EMSTAC TA Liaisons. 

LA-Exchange (the EMSTAC Listserv) 

In one year, the number of registrants to the EMSTAC Listserv increased by 58 percent to 95 
subscribers. Remarkably, this figure represents a 171 percent increase in the number of subscribers 
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since the first year that EMSTAC started to use LA-Exchange as a technical assistance support. 
Importantly, the nature and quantity of the information for which the Listserv is used have increased. 

Approximately 229 messages have been posted to LA-Exchange this year, a 1 16 percent 
increase from last year. Topics for discussion included such items as assessment, assistive 
technology, behavior programs, disproportionality, ESL, educational research, IDEA, and NCLB. 
The Listserv proved to be an efficient way to disseminate information to EMSTAC LAs, since 69 
percent of the individuals subscribed to LA-Exchange are LAs or have direct connections to local 
school districts. Therefore, these individuals are able to disseminate materials generated through LA- 
Exchange to a wider audience of educators at the local level. For instance, in response to resources 
posted to LA-Exchange about No Child Left Behind, one LA commented, “Thanks! I have passed 
this on to other interested people in the building. ” 

A distribution of the nature of these posts is indicated below; 

• Resource Announcement - (34 percent) 

• Response to a post: editorial comment - (1 percent) 

• LA-exchange administration announcement - (47 percent) 

• Resource Request - (2 percent) 

• Response to a resource post: - (7 percent) 

• Other posts - (8 percent) 

Although EMSTAC LAs have not consistently used LA-Exchange as an ongoing forum for 
dialog, the other support mechanisms available, such as the threaded discussions and chat events, 
have had increased participation, which may explain the dialogue that occurred on the Listserv. 
Additionally, we learned that LAs on occasion follow-up their discussion of a topic posted on LA- 
Exchange with their TA Liaison privately or contact other LAs to discuss the topic offline. This has 
occurred regarding such topics as disproportionately and computerized individualized education 



O 

ERIC 



m-21 



American Institutes for Research 



programs, where LAs have used LA-Exchange as a forum to learn about each others’ interests, and 
continued their discussion via electronic mail and telephone outside of the EMSTAC open support 
forums. 



The EMSTAC Electronic Mailbox 

The general delivery address for the EMSTAC mailbox received 136 posts this year, with the 
number of messages related to a request for resources nearly doubling from last year and representing 
approximately 30 percent of the total contacts via the EMSTAC mailbox. Electronic communication 
via the EMSTAC mailbox continued to be a timely and efficient communication and support 
mechanism. The mailbox served a multitude of purposes: 

^ As recruitment mechanism through which EMSTAC staff can answer questions posed by 

prospective LAs and can receive interest forms submitted by LA candidates. Seventeen inquiries 
via the electronic mailbox have been received to date, for this purpose. 

> To support the work of LAs as they move through the on-line training program, and answer 
questions related to the content and process of the instruction. To date, three inquiries have been 
received related to training. 

> Asa forum through which LAs can communicate with EMSTAC TA Liaisons regarding their 
work. We have received a total of 10 LA Journal submissions via the EMSTAC mailbox during 
this evaluation period. 

> Asa venue to share resource information and to access EMSTAC products and materials. To 
date, over 41 pieces of correspondence were received regarding upcoming conferences, 
commercial resources, and requests for EMSTAC products and resources related to 
disproportionality and behavioral interventions. We received over 25 e-mails from parents of 
children enrolled in elementary and middle schools requesting resources and support. These 
persons were directed to more appropriate technical assistance providers such as the National 
Information Center for Children and Youth with Disabilities (NICHCY) and the Technical 
Assistance Alliance for Parent Centers - the Alliance (PACER Center). 



Computer-based Videoconferencing 

EMSTAC took advantage of the advances made in telecommunications technology by 
implementing a computer-based videoconferencing program. We implemented the EMSTAC 
Videoconferencing Carousel as a means to connect educators and their classrooms across the nation. 
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Using this platform, participants are able to see and hear others in their own environments, creating 
two important strengths of this technology. First, computer-based videoconferencing has enabled 
participants to showcase specific initiatives in their classrooms. This “pride of place” happens 
because participants are in their own setting, often surrounded by colleagues and students. Second, 
this technology provides a platform that creates a level playing field among participants and an 
expert lecturer who may facilitate the event. There are no power struggles using this technology, as is 
often the case when expert leaders facilitate professional development. Computer-based 
videoconferencing helps everyone feel as if they are equal contributors to the event; therefore, they 
may be more apt to participate and offer their experience. The following was taken from the 
EMSTAC website and identifies further advantages of using this technology. 



Come Jump On 

EMSTAC's Videoconferencing Carousel! 

3L 



EMSTAC uses computer-based videoconferencing to provide its partners (local school district educators) with 
opportunities to interact with their peers, participate in professional development forums, and to share their 
experiences related to specific school-based interventions (such as literacy programs). This interactive forum 
holds the following advantages over asynchronous types of communications as well as ISDN based 
videoconferencing systems. The advantages of using this technology include: 

• Participants are able to see, hear, and interact with their colleagues through live video and audio functions. 

• The medium is an Internet based technology, thus providing multi-point access for up to six participants 
during any given event. 

• Conference attendees can use interactive features such as a chat board, white board (file development), and 
application sharing (participants can simultaneously share applications such as Word, Excel, and 
PowerPoint). 

• The costs for the end-user are minimal. 

We offered computer-based videoconference events in the following topics: 

<♦ Peer Assisted Learning Strategies 
<♦ Motivating and Sustaining Literacy Growth. 

❖ Literacy. 
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TA LIAISON SUPPORT 



TA Liaison Support 



As indicated in previous reports, TA Liaisons from the EMSTAC staff are matched with 
district LAs based upon the needs of the district and the expertise of the TA Liaison. Therefore, the 
support provided by Liaisons, regardless of how this support is delivered is within the experiential 
capacity of our staff. This alignment creates a support culture through which LAs can feel confident 
that they are working with an experienced and competent professional. We believe this factor is 
important in establishing and maintaining a respectful and trusting relationship between TA Liaisons 
and LAs. 

In this section we will describe the nature and method of the support provided by TA 
Liaisons to LAs. TA Liaisons were asked to complete weekly logs as a means of collecting data 
regarding their support; therefore, we are able to characterize the nature and kind of interactions and 
support common in these relationships. The following outlines the kinds of support activities that 
were most frequently rendered by TA Liaisons. An example of the nature of this support is included 
for each. 

TA Liaisons were asked to describe their TA activities according to distinct domains of work. 

> Research for LA: A district is seeking to implement programs regarding alternatives to 
suspension. This comes after the State Board of Education has taken over administrative 
control of the local school district. The LA scanned research-based publications and used 
the expertise and experiences of her colleagues to compile information regarding 
alternatives to suspension. The LA shared this information with her supervisors and 
colleagues in their work to develop an alternative program based upon research and best 
practices in the field. 

> Troubleshooting/Problem-Solving: An LA changed positions within the school district. 

As the new professional responsible for Title I activities, the LA expressed concern about 
the relationship between legislation impacting on Title 1 (No Child Left Behind) and 
special education legislation (IDEA). T^e TA Liaison collected information regarding the 
overlap between these two laws and specifically those aspects of the legislation that have 
implications for students with disabilities. These resources were shared with the LA by 
the Liaison who continued to assist the LA as he acclimated to his new position. 

> Update from District: Local school districts do not function in a static environment. There 
are often administrative, managerial, and program changes that necessitate our being 
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flexible and responsive to their changing conditions. This was the case in one district 
where managerial changes forced the EMSTAC LA into a new position, which precluded 
her from continuing in the role of Linking Agent. The TA Liaison worked with district 
personnel and the outgoing LA to identify new professionals most appropriate for the 
role. It was important for the TA Liaison to orient and train the new representatives to 
facilitate their understanding and success in the LA role. 

>• Evaluation Activities: In one situation a TA Liaison is working with an LA to evaluate 
their reading initiative, the LA requested materials and support in developing a protocol 
and implementing a procedure for evaluation. This same TA Liaison worked with the 
district to coordinate the interview and data collection for the EMSTAC Annual 
Evaluation. 

> Administrative Activities; Sometimes there are technical and administrative activities that 
characterize the support. For instance, on occasion TA Liaisons work with their LAs to 
help them get online by assisting them to re-register on the EMSTAC restricted-access 
web site. For example, LAs may forget their usernames and need help to re-register. 

> Other TA Activities: The TA Liaisons and several LEAs works closely with partner 
organizations such as the 100 Black Men of America and the National Association of 
State Directors of Special Education to coordinate TA activities related to 
disproportionality. The collaborative events and state meetings and conferences in which 
she participates help to facilitate the coordination of technical assistance by these various 
providers. 

Our goal in all of our support activities is to build local school capacity. As described in this 
section, we use a variety of platforms to ensure that LAs access our support and materials. Support 
forums such as chat events, threaded discussions, electronic mail, and computer-based 
videoconferencing ensure that our LAs are accessing our materials. However, the ongoing support 
characterized by the relationship between the TA Liaison and the LA serves to ensure that these 
materials are not only accessed, but they are being used in a way that ultimately benefits children 
with disabilities. The distance education methodologies primarily used in Strategy III appear to have 
strengthened the quality of relationships between TA Liaisons and LAs, as evidenced by the types of 
comments received from LAs throughout the year, some of which are included in this section. There 
may be several factors that affect this relationship including the efficiency and quality of electronic 
forums, our increased expertise and knowledge of these electronic forums, and our ability to 
accurately match the needs of our districts with the capabilities of our staff We have taken advantage 
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of electronic forums in our work to build capacity at the local level, capacity that starts at the at the 
LA level as they access and use TA support. The process is reflected in the following LA feedback, 
“Also, thanks for the packet of resources that came with the letter. It was wonderful. You guys are 
such a wealth of information! You have provided me with much and have saved me so much time. ” 



O 



m-26 



American Institutes for Research 



STRATEGY COMPARISONS AND 
PREDICTOR VARIABLES 



Strategy Comparisons and Predictor Variables 



Introduction 

This section of Chapter III reports findings for the 32 districts from which we collected data 
during the 2001 - 2002 school year: seven Strategy I districts, nine Strategy II districts, and 16 
Strategy III districts. The results presented in this section cover three areas. First, general data are 
presented regarding the demographics of the 32 districts. Second, data regarding the progress of each 
district in their efforts to implement research-based practices are presented, as well as data regarding 
the differences between each of the three Technical Assistance Strategies. Finally, data are presented 
indicating those variables that appear most important to the change process. Although some of these 
data were presented in previous reports, data from the current year represent the most complete data 
set based upon the largest number of districts. 

Demographic Factors 
District Size 

Student enrollment across the districts ranges widely, from 869 to 735,058 (see Exhibit III-7). 
Average enrollment is 72, 619 students. Enrollment within the different Strategies also varies 
widely. For Strategy I, enrollments range from 1,216 to 735,058 students with an average of 139,843 
students. In comparison. Strategy II districts range from 1050 to 435,470 students, with an overall 
average enrollment of 83,049 students. District enrollment in Strategy III averages 48,799 students, 
with a low of 869 students to a high of 244,000 students. Despite variation in size among and within 
the strategies, differences among the three strategies, with respect to size, were not statistically 
significant. 
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EXHIBIT MI-7 



Summary of District Demographic Characteristics (n = 32) 



District 


Size (Number of 
Students) 


% Special 
Ed 


% 

Minority 


% Free/ 
Reduced 
Lunch 


STRATEGY I 






.• 




District 7 


735,058 


11.3 


80.7 


72.8 


District 1 


10,416 


18.1 


4.8 


59.0 


District 5 


36,885 


11.5 


13.6 


24.0 


District 3 


18,502 


12.3 


15.5 


20.2 


District 4 


1,216 


11.4 


4.7 


10.4 


District 2 


167,704 


8.0 


96.3 


70.0 


District 6 


9,750 


3.9 


82.3 


30.2 


Strategy I Mean and Standard Deviations 


139,843 

(268,712) 


10.9 

J4.3) 


42.6 

(41.5) 


40.9 

(25.7) 


STRATEGY II 






..n,’ ■ •'/' ■>•■ 

■ 'V' : \ . . 


: r 'T ' . 


District 15 


2,953 


10.4 


77.75 


72.38 


District 12 


57,636 


13.3 


95.7 


85.4 


District 1 1 


55,660 


11.4 


70 


62 


District 13 


17,061 


9.8 


6.2 


11.3 


District 10 


1,050 


24.2 


55 


73.6 


District 16 


3,369 


13.6 


2.9 


6.3 


District 9 


435,470 


11 


85.6 


90.4 


District 14 


66,918 


10.6 


52.9 


31.6 


District 8 


107,322 


12.4 


40.3 


29.3 


Strategy II Mean and Standard Deviations 


83,049 

(137,062) 


13.0 

(4.4) 


54.0 

(32.9) 


51.4 

(32.1) 
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District 


Size (Number of 
Students) 


% Special 
Ed 


% 

Minority 


% Free/ 
Reduced 
Lunch 


STRATEGY III 




■ ' . 




• • r ‘ ' 


District 24 


1,374 


10 


7.0 


7.2 


District 29 


3,923 


10.2 


88.5 


8.5 


District 19 


74,491 


13.5 


24.9 


26.0 


District 17 


30,655 


14.6 


50.1 


53 


District 23 


244,000 


11 


45.8 


36.5 


District 18 


49,368 


14.7 


40 


- 


District 20 


17,606 


- 


14.6 


28.1 


District 2 1 


23,000 


9 


40 


- 


District 25 


22,466 


17.4 


94.3 


68.8 


District 26 


133,336 


10.8 


89.7 


70.9 


District 27 


4,360 


11.4 


34.6 


31.4 


District 28 


13,507 


15 


99 


91 


District 30 


57,621 


13.7 


95.8 


93.3 


District 3 1 


2,200 


18 


100 


86 


District 32 


869 


15.9 


23.4 


- 


District 22 


102,013 


11.3 


53.0 


36.1 


Strategy III Mean and Standard Deviations 


48,799 

(65,030) 


13.1 

(2.8) 


56.4 

(32.9) 


49.0 

(30.2) 


GRAND MEAN and Standard Deviations 


78,930 

(151,516) 


12.6 

QJl 


52.7 

(34.2) 


47.8 

(29.1) 



Percentage of Students Receiving Special Education Services 

As seen in Exhibit III-7, the proportion of students receiving special education services 
across strategies was comparable. The average, across districts, was 12.6 percent, with 10.9 percent 
of students in Strategy I receiving services, 13.0 percent of students in Strategy II districts receiving 
them, and 13.1 percent of students in Strategy III districts receiving special education services. 
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Percentage of Minority Students 

The districts currently working with EMSTAC enroll an average of 52.7 percent minority 
students. As seen in Exhibit III-7, the range in minority students across all three TA strategies is 
from a low of 2.9 percent to a high of 100 percent. Though this represents a wide range, the 
differences among the strategies, with respect to proportion of minority students, were not 
statistically significant. 

Percentage of Students Eligible for Free or Reduced-Price Lunch 

As a measure of socioeconomic status, EMSTAC collected data on the proportion of students 
within each district who are eligible for free or reduced-price lunch. Overall, 47.8 percent of the 
students in districts working with EMSTAC are eligible. Slightly more than one third of students in 
Strategy I are eligible, more than half of the students in Strategy II districts are eligible, and 
approximately half of the students in Strategy III are eligible. These differences were not 
statistically different. 

Tracking District Progress - Linking Agent Implementation 

We measured the degree to which LAs have engaged in activities that facilitate the adoption 
and implementation of research-based practices within their school districts by rating each district, 
using the LA Implementation Scale. The scale contains 27 possible questions, each of which deals 
with critical activities that LAs may engage in to facilitate the change process. Each of the first 18 
questions is based on a scale of 1 to 3. The 19'*’ item is a binary variable (coded 0 or 1). The 
remaining 8 questions are based on a scale of 0 to 2. The resulting possible range of scores on this 
scale was 18 to 71 although, ultimately, since districts are not limited in the number of new initiatives 
they can pursue, there is no upper limit on the range of scores. 
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The sources for the ratings primarily included the LA interviews and the TA Liaison 
interviews (see Appendix B). To ensure that ratings were made in a reliable manner, two EMSTAC 
staff members independently reviewed the relevant data and scored each item. Inter-rater reliability 
was above .70 on four randomly chosen LA Implementation Scales. This year, scoring procedures 
were revisited by one of the raters who scored the LA Implementation scale last year. 

In Exhibit III-8 below, points in time corresponding with data collection for each TA 
Strategy are provided. It should be noted that attempts were made to ensure that the number of 
months elapsed since beginning work with EMSTAC was held constant. However, some variation 
occurred in the time of these data collection points due to circumstances beyond the control of 
EMSTAC. The following ranges for each data collection point are also presented in Exhibit III-8. 

EXHIBIT III-8: 

Available Data for LA Implementation Scale 





Year 1 


Year 2 


Years 


Year 4 


Range 


(8-13 months) 


(20 - 23 months) 


(31-32 months) 


(40) 


Strategy I 


Spring 1999 


Spring 2000 


Spring 2001 


Spring 2002 


Strategy II 


Spring 2000 


Spring 200 1 


Spring 2002 




Strategy III 


Spring 2002 









Comparisons Over Time and Between Strategy Types 

We compared districts across strategies with respect to LA Implementation Scale scores. As 
Exhibit III-9 shows, during the first year of implementation, the districts in Strategy I achieved a 
mean scale score of 37.6, whereas districts in Strategy II achieved a comparable mean of 37.1, and 
districts in Strategy III achieved a mean score of 35.9. Data analysis failed to reveal a statistically 
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significant difference between the three TA strategies on the LA Implementation scale after one year 
of working with EMSTAC. 



EXHIBIT MI-9; 

Summary of LA Implementation Scale Scores 



Year 1 Year 2 Year 3 Year 4 



Strategy I 


37.6 (6.8) 


46.1 


Strategy II 


37.1 (6.7) 


43.0 


Strategy III 


35.8 (8.0) 


n/a 


Grand Mean 


36.8(7.1) 


44.5 



(2.8)** 


50.7(7.4) 


52.1 (11.9) 


(8.1)* 


43.5 (6.9) 


n/a 




n/a 


n/a 


(5.5) 


50.7(7.1) 





Note: Numbers in parentheses are standard deviations. 

♦p<.10. 

** p < .01. 

By the second year, districts in both Strategies I and II increased their scale scores 
significantly (t = 4.33, p < .01; t = 2.26, p < .10, respectively). Districts in Strategy I averaged 46.1, 
and districts in Strategy II averaged 43.0. Despite the slightly greater increase in growth for Strategy 
I districts during Year 2, the difference in the LA Implementation score between Strategy I and II 
districts was not statistically significant. 

In Year 3, Strategy I districts jumped to an average score of 50.7 while Strategy II districts 
increased their mean scale score to 43.5. The scale score gain fi"om Year 2 to Year 3 was not 
significant for either Strategy. In addition, the difference between Strategy I and Strategy II districts 
was not great enough to be considered statistically significant during Year 3. 




m-32 



n 



3 



American Institutes for Research 



Finally, during Year 4, Strategy I districts earned a scale score on the LA Implementation 
scale of 52.1, documenting continued growth. Similar to the previous two years, the increased score 
did not reflect a statistically significant gain. 

These findings indicate that districts within both Strategies I and II have made noticeable 
progress in terms of LA implementation of activities that support research-based practices within 
their districts. Although statistically significant gains could not be detected each year, the small 
sample sizes within each strategy may have made such gains difficult to detect. 

Just as important as the constant gains that have been found within each strategy, results of 
the analysis indicate that Strategy I and Strategy II districts have not differed in their on-going 
implementation of the LA activities over three to four years of the project. Further, after one year. 
Strategy III districts progressed similarly to their Strategy I and II counterparts. The following 
exhibits (Exhibits III- 10, III-l 1, and III- 12) illustrate individual district scores across years. Because 
Strategy III districts had only completed one year of implementation at the time of the analysis, those 
districts are only compared on Year 1 . Strategy I and II districts are compared across three years. In 
addition. Exhibits III- 13 and III- 14 further illustrate district scores within Strategy I and Strategy II 
over the course of the project. 

Important Predictors of Linking Agent Implementation 

It is important to study the factors that influence LA implementation in order to develop 
external and internal technical assistance systems of support that will maximize the potential impact 
of these variables. To better inform EMSTAC’s technical assistance efforts, we looked at the 
relationship between LA Implementation (for each year of data collection) and a number of factors. 
Before reviewing these factors, it is important to note that the analyses conducted to identify these 
variables aggregated data across districts from all three TA strategies; therefore, the findings 



O 



m-33 



American Institutes for Research 



Comparison of Strategy I, II, and III Districts on LA Implementation Scale Year 1 Only 
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Comparison of Strategy I and Strategy II Districts on LA Implementation Scale Year 2 Only 
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EXHIBIT MM2: 
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EXHIBIT IIM3: 

Comparison of LA Implementation Scores Over Time: Strategy I Only 



80 

70 




District 4 District 3 District 1 District 6 District 2 District 5 District 7 



□Year 1 
□Year 2 
□Years 
□Year 4 
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EXHIBIT 111-14: 

Comparison of LA Implementation Scores Over Time: Strategy II Only 




District 15 District 12 District 11 District 13 DstrictIO District 16 Districts Dstrict9 Districts 
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discussed next are not TA strategy specific. The important predictor variables that were examined 
included: 

• Perceived level of building support for the LA and the program, 

• Perceived level of district support for the LA and the program, 

• Extent to which teachers agreed with the chosen intervention and believed that it would 
be successful, 

• The locus of change for addressing the need (external, community or state-driven versus 
internal, school or district-initiated), 

• Whether the initiative or program was voluntary or mandated by the district or state, 

• The existence of other, past initiatives (past 5 years) in special education, 

• The existence of financial resources other than EMSTAC (if applicable) to support the 
initiative, 

• The extent of internal collaboration between the LA and teachers and administrators 
within the school district, 

• Level of support (non-financial) from external agents other than EMSTAC, 

• Perceived district openness to outside technical assistance, 

• Perceived building-level openness to outside technical assistance, 

• Type of TA needs (e.g., reading, discipline, inclusion, and so on), and 

• Reason for becoming involved with EMSTAC. 

As expected, not all of these factors were related to LA implementation. The factors that were 
significant are discussed next. 

During the first year of implementation, district-level support for the initiative was a 
significant factor (x2 (2) = 4.9 ,p< .10). When district-level administrators were at least moderately 
involved in the initiative, providing periodic input and making decisions that supported the goals of 
the project, then implementation of LA activities was enhanced. The contributing role of district level 
support in subsequent years, however, was not related to implementation of research-based practices. 
This suggests that during the first year in particular, support provided by district administrators is 
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critical to ensuring the legitimacy of a new initiative. This legitimacy is likely to ensure that 
resources are dedicated to the project and that staff are made aware of the importance of the project. 

A second important factor contributing to the progress that a district makes in the first year of 
working with EMSTAC is the “locus” of the initiative (%2 ( 2 ) = 5.4, p<.10) that is being implemented. 
Specifically, when the motivation to implement a particular initiative began at the school level, based 
upon the school’s recognition of a particular need, then LAs were more engaged in activities 
supporting the school/district in their efforts to address the need. 

A third factor related to the progress that LAs were able to make in supporting research- 
based initiatives was the district’s history with past initiatives. In particular, districts that progressed 
the furthest during their first year of partnering with EMSTAC reported a history of implementing 
other small-scale initiatives within the past five years (^2 ( 2 ) = 6.0, p<.05). One interpretation of this 
finding is that these districts have developed a systemic value that supports and encourages the 
implementation of new initiatives. In addition, LAs and schools may benefit from their previous 
experience with other initiatives and put their lessons learned to work in implementing their 
EMSTAC-related initiatives. Unfortunately, our data do not indicate how successful these districts 
were in implementing past initiatives. 

During the second year of working with EMSTAC the only significant predictor variable that 
emerged was internal collaboration (^2 ( 2 ) = 6.3, p<.05). LAs who engaged in regular on-going 
consultation and discussion with teachers, administrators, and other school staff were most successful 
in implementing activities supportive of research to practice. The importance of this variable during 
the second year of work with EMSTAC suggests that collaboration is particularly critical as districts 
begin initial implementation and on-going support of the program (typically during the first year 
districts engaged in planning activities). 
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When comparing results from last year’s preliminary data analyses, it is clear that fewer 
variables were detected as being important during the second year of implementation. For example, 
results from prior evaluations indicated the importance of building-level support. Results from this 
year’s complete data set failed to indicate the significance of this factor. This finding is difficult to 
explain but suggests that some of the districts have developed the capacity to proceed without active 
involvement from building administrators. This is consistent with the concept that leadership is most 
critical in the early stages of implementation and change when an initiative is being launched, but is 
less important when capacity and motivation has been sufficiently developed for the program to 
operate and be sustained on its own. An alternative explanation relates to the pattern that building- 
level support across school sites has increased over the years of the project to the point that there is 
little or no variation. This diminishes the power of this variable to predict trends in implementation. 

Finally, during Year 3 of work with EMSTAC, the variable “Reason for Working with 
EMSTAC” was found to be significant {yl ( 2 ) = 6.4, p<.10). Districts that believed EMSTAC would 
be a beneficial resource, and therefore decided to partner with EMSTAC, had the highest LA 
Implementation scores. It is likely that these districts value the input of outside technical assistance 
entities and are concretely invested in implementing change. As a result, these districts have been 
able to progress on their chosen initiatives and utilize EMSTAC in the process. In essence, this 
finding suggests that those districts that initially believed in the potential benefits of EMSTAC were 
most likely to have continued to strive toward impacting change and “stick with” the process. 

EMSTAC Factors 

We also examined the potential relationships between LA implementation and EMSTAC 
support factors, including: 

• whether the district had chosen an intervention or program prior to working with 
EMSTAC, 
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• what type of support EMSTAC supplied, 

• how much support was received from EMSTAC (i.e., number of monthly contacts and 
number of annual visits from the TA Liaison) 

Results of correlational analyses revealed the importance of two EMSTAC factors. First, the 
number of TA Liaison contacts that EMSTAC had with a district, during a district’s first year of 
collaboration with EMSTAC, was positively related to LA progress during Years 1 and 2 (r = .49, 
p<.10; r= .64, p<.05). Consistent with findings reported previously, this finding suggests the 
importance of personal contact between the TA provider and the local school district, at least 
initially. The second important factor was the number of site visits made to a district. Specifically, 
the number of site visits during Year 1 of work with EMSTAC was positively related to LA 
Implementation scores in Years 2 and 3 (r = .59, p<.03; r = .85, p<.01). The findings regarding 
contacts and site visits are notable because they suggest that support during the first year is related to 
positive changes in subsequent years. Although the long-term implications of these findings cannot 
be determined, these results preliminarily suggest that more intensive support during the first year of 
support may result in greater district capacity to sustain and replicate the process of change in the 
future. 

Stage of Change 

In addition to exploring the level of LA implementation and the factors that contribute to 
successful implementation, we also studied the change process more globally. Each year, districts 
working with EMSTAC are classified as being at a specific stage of change according to Havelock’s 
Change Cycle. The cycle has seven stages: Care, Relate, Examine, Acquire, Try, Extend, and 
Renew. These stages approximately indicate where a district may be within the process of change. 
We assigned each district a score from 0 to 6, corresponding to each level in Havelock’s Change 
Cycle (Care = 0, Relate = 1, Examine = 2, Acquire = 3, Try = 4, Extend = 5, and Renew = 6). The 
specific stage of change for each of the 32 districts is provided Exhibit III - 15 below. 
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EXHIBIT MM 5: 

Summary of Havelock Stage by District 



Site 


District Havelock Stage 


STRATEGY I 




District 1 


Extend ' 


District 2 


Try 


District 3 


Extend 


District 4 


Extend 


District 5 


Renew ^ 


District 6 


Extend 


District 7 


Extend 


STRATEGY II 




District 8 


Try and Extend 


District 9 


Try 


District 10 


Try 


District 1 1 


Extend 


District 12 


Extend ^ 


District 13 


Extend 


District 14 


Try 


District 15 


Try 


District 16 


Acquire 


STRATEGY m 




District 17 


Try 


District 18 


Acquire and Try 


District 19 


Examine 


District 20 


Try and Examine 


District 21 


Relate and Examine 


District 22 


Acquire 


District 23 


Try and Extend 


District 24 


Extend and Examine 


District 25 


Examine and Acquire 


District 26 




District 27 


Try 


District 28 


Extend 


District 29 


Try 


District 30 


Examine ^ 


District 31 


Try 


District 32 


Acquire 



‘ This district is in its third topic. 



^ This district is in its second topic. 



^ This district is in its second topic. 



"^This district is in its second topic. 
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Strategy I Districts 



As can be seen in Exhibit III- 15, two of the seven Strategy I districts have completely cycled 
through Havelock’s stages of change. For these two districts, more than one need has been identified 
and addressed during the four years in which they have worked with EMSTAC. Another four of the 
Strategy I districts are currently making efforts to extend and sustain research-based practices that 
were implemented to address a need identified during their first year of work with EMSTAC. These 
districts have not yet identified a new need to address. Finally, one district is in the “Try” stage and 
has been in that stage since Year 2 of the project. Although efforts were made to extend their chosen 
initiative to other buildings, barriers prevented this fi'om fully occurring. As a result, efforts in this 
district have remained on implementing the initiative in the originally identified building. 

Strategy II Districts 

Data were collected fi'om nine Strategy II sites during the 2001-2002 school year. Five of 
these districts have been working with EMSTAC for three years. Four of these five districts can 
currently be characterized as falling in the “Extend” stage of change. These four districts have 
expanded implementation of their initially identified practices beyond the original site (school). The 
remaining four Strategy II districts have worked with EMSTAC for two years. Three of these 
districts are in the “Try” stage of Havelock’s model, suggesting that they are implementing research- 
based strategies in their target schools or classrooms. The fourth district is in the “Acquire” stage, 
indicating that this district is still planning implementation of the initiative after two years. 

Strategy III Districts 

Finally, ten of the sixteen Strategy III districts that began work with EMSTAC this year are 
in the ‘Try” stage of change. These eight districts have begun implementation of research-based 
practices in an effort to meet their identified need. Two of these ten districts began implementation 
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prior to EMSTAC’s involvement and subsequently these two districts are actually in the “Extend” 
stage of implementation. Four of the Strategy III districts were in the “Examine” stage, indicating 
that they were further exploring their identified needs and/or exploring solutions. The remaining two 
Strategy III districts are in the “Acquire” stage. 

Consistent with previous analyses, the “stage of change” that a district has reached appears to 
be largely a function of the time that the district has worked with EMSTAC. For example, the 
majority of Strategy I districts, now finishing their fourth year of work with EMSTAC are in the final 
stages of Havelock’s cycle or they have repeated the cycle with new needs being addressed. The 
majority of Strategy III sites, on the other hand, are in the early to mid-stages of Havelock’s cycle. In 
last year’s evaluation, it was demonstrated that despite initial differences between Strategy I and 
Strategy II districts, these differences disappeared over time. Similarly, the mean “Havelock” score 
for Strategy III districts closely approximates Strategy II districts after one year of work with 
EMSTAC (Strategy III = 3.75; Strategy II = 3.7). It is possible, however, that after two years of work 
with EMSTAC the difference between Strategy III and Strategy I sites (Strategy I sites = 4.7) would 
disappear. 

A second important trend that is important to note when comparing Strategies I, II, and III 
relates to the identification and addressing of new needs. Ensuring that each district identified and 
began the process of implementing solutions to address a second (or third) need was an important 
goal for EMSTAC. Emphasis was placed on this goal because it was believed that it would 
demonstrate a district’s capacity to independently address identified educational needs using 
research-based practices. Only two Strategy I districts were able to meet this goal. Similarly, only 
one Strategy II district met this goal and one Strategy III district met this goal. Observation of 
longitudinal results appears to indicate that most of our districts hit a plateau over time (around the 
third year) and did not “renew” their efforts to identify new needs. A number of reasons are likely 
responsible for this, but within the current discussion the critical point is that “TA Strategy” is 
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unlikely to play a critical role in determining whether a district identifies a new need to be addressed. 
Potentially, if support for a district remained at an “intense” level even into the third or fourth year of 
partnership then districts may have had more motivation to address a new need. 

The results presented here are not intended to suggest that districts do not progress in the 
absence of on-going “intense” support; conversely, the majority of districts continued to make 
progress and support research-based initiatives, but their focus was on sustaining and broadening the 
impact of the initially identified intervention. Future research might investigate whether the variables 
that influence sustainability of one practice are distinct from those that influence renewal activities 
(re-starting the cycle). 

Overall, these findings confirm that the districts with which EMSTAC is working are varied 
and diverse. However, there are no statistical differences between the strategies with respect to 
demographic variables. 

In addition, these findings have shown that the strategies are comparable with respect to their 
level of LA implementation. However, we are encouraged by what appears to be real and significant 
improvement in districts using either Strategy I or Strategy II over time. Through three (Strategy II) 
and four (Strategy I) years of data collection, districts continue to progress in their efforts to support 
implementation of research-based programs and strategies. 

What factors are driving this progress? The answer appears to be a combination of internal 
and external factors. Beginning with the driving force behind the initiatives, it is clear that when 
schools themselves initiate programs, instead of accepting a district or state mandate, they have 
achieved more success in implementing the programs. However, active district level support of a 
chosen initiative also appears to be critical in ensuring implementation. During Year 2 of work with 
EMSTAC, collaborating closely with colleagues helps LAs have an impact on the staff while 
gamering support and buy-in. Finally, during Year 3, it appears that districts that initially believed 
that EMSTAC would be a beneficial resource were most likely to progress in their change efforts. 
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This finding may reflect the relationship between open systems that value input from outside sources 
and an orientation toward continuous improvement. 

Technical assistance also plays a key role in moving districts toward implementation of 
research-based programs. Districts visits by TA Liaisons and phone calls to support LAs, 
particularly during Year 1, appear to help districts achieve greater levels of success. 

District success relies on a number of factors both internal to and external to the district. It is 
clear that LAs who have been most successful in initiating and sustaining change have collaborated 
with others and have relied on strong district level leadership. Further, the most successful districts 
have implemented initiatives that are largely driven at the school level, and these districts appear to 
see the potential benefits of working with a technical assistance provider such as EMSTAC. In fact, 
early, frequent contact and support from the technical assistance provider appear to have significant 
benefits for districts over a two to three year period. 
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IMPACT 



Impact 



Impact refers to the degree to which a program such as EMSTAC brings about the desired 
change in a particular process, structure, or population such as LAs, teachers, or school children. 

This section of Chapter 3 provides an overview of EMSTAC’s impact through a review that consists 
of the following subsections: 

• Local accomplishments and supports, 

• EMSTAC and LA relationships across strategies, 

• Results of EMSTAC supports and relationships with LAs, 

• Relationships between LAs and teachers, 

• LA impact, and 

• Child outcome data. 

These subsections provide a review that walks through the various linkages that connect EMSTAC to 
different interconnecting outcomes, moving from the initial EMSTAC effect on local LAs, to LA 
impact on teachers, and finally child outcome data. 

Throughout the life of this project EMSTAC has been interested in the impact of its services 
on building local capacity to facilitate change and move research-based practices into schools and 
classrooms across the three different TA strategies. The project is also interested in how these 
activities affect student performance, though the primary focus has been directed to the process of 
moving research to practice. Research-based practices have already been validated through other 
studies focused on measuring the impact of these practices on child outcomes. 

In order to assess impact at each level it is important to step back and see the logical chain 
through which EMSTAC services would affect outcomes. Generally, each technical assistance 
strategy utilizes the relationships between an EMSTAC TA Liaison and an individual working in the 
school district (Linking Agent). Through the Linking Agent, EMSTAC supports and brings resources 
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to bear on educational needs that are locally identified. Thus, these TA supports are utilized in an 
effort to bring about school change and implementation of research-based practices. This process is 
reflected in the TA Logic Model (see Exhibit III- 16). 

The logic model reflects the process by which EMSTAC services are made available to local 
school districts. Importantly, the logic model does not change based upon the particular TA strategy 
being examined. As can be seen in the model, there are a number of steps that must be realized in 
order for technical assistance supports to have impact at the student level. EMSTAC’s evaluation 
process focuses on each of these steps and the relationship between them. Further, although not 
represented here, EMSTAC examined a number of “extraneous” variables that do not directly relate 
to the provision of technical assistance but do influence a school or district’s ability to implement 
change. 



EXHIBIT 111-16: 
TA Logic Model 
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Drawing from this logic model we can see that once a relationship has been established 
between EMSTAC and a district, EMSTAC focuses on providing individualized support to help the 
districts move research to practice. This occurs through LA training, technical assistance, and a wide 
variety of products. These activities involve a two-way exchange where the EMSTAC TA Liaison 
learns from the district about their needs, while at the same time building the district’s capacity to 
identify those needs. 

Once the district’s needs are identified, EMSTAC is designed to connect the district LA with 
research-based practices and services tailored to their needs, and to support the LA in their effort to 
engage in activities that will facilitate implementation of these practices. The LA Implementation 
Scale includes items that reflect activities LAs are likely to engage in if they are involved in 
facilitating change within their district. Examples of these items include: 

• conducts a needs assessment, 

• evaluates potential research-based intervention programs, and 

• assists teachers in implementation. 

These activities are designed to help teachers and other service providers put research-based 
practices into the school, classroom or whatever setting is appropriate. Once this is achieved, 
research-based practices connect with students and hopefully improve their learning and behavior. 
From this perspective we can see how EMSTAC ultimately impacts child outcomes through a logic 
chain that involves a number of important links. Within this logic chain the primary contribution of 
EMSTAC is on building the capacity of the LA to support effective implementation of research- 
based practices in schools and classrooms. This is the critical channel through which EMSTAC 
eventually impacts student outcomes. 
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Local Accomplishments and Supports 

Examination of EMSTAC’s impact at the local level can be understood when considering 
three major objectives that EMSTAC has accomplished: 

• EMSTAC has increased the importance that professionals working in local education 
agencies place on research-based practices. 

• EMSTAC has demonstrated that information regarding research-based practices are 
permeating the local level. 

• EMSTAC has documented that positive change (i.e., the implementation of research- 
based practices) is actually occurring in the districts that are served. 

EMSTAC was designed to accomplish these objectives by providing LAs with access to a 
number of resources highlighting the use of educational programs and practices that are empirically 
valid. This has been accomplished through LA training, on-going support of LAs, the EMSTAC 
Listserv, Chat Sessions with researchers, and through the use of a web-based bulletin board that 
promotes exchange of ideas. Use of these services is provided in the description of EMSTAC 
supports and usage data. 

EMSTAC and Linking Agent Relationships Across Strategies 

When looking closely at the support EMSTAC provides school districts, it is difficult to 
generalize about the nature of the relationships between EMSTAC and LAs. Even within the three 
Strategies designed to serve as the frameworks of EMSTAC support, there are unique circumstances 
in each district that demand flexibility and some variation in the kind of support EMSTAC provides 
to districts and their LAs. This section explores the evidence that describes the nature of technical 
assistance and EMSTAC support that is occurring in these unique and individual systems, and the 
resulting relationships between EMSTAC and district LAs. 

Due to the uniqueness of school districts and their needs, EMSTAC has become versatile and 
flexible in its approach to providing technical assistance (TA) to LAs around the nation. Although 
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there are three strategies that serve as the support foundation, TA Liaisons and EMSTAC support 
staff have found that modifications and a “whatever it takes” attitude are necessary to achieve the 
Center’s goals and meet the needs of participating districts. Despite their initial difference in design, 
there are some notable similarities within and differences among strategies with respect to the kind of 
EMSTAC support LAs use and the subsequent nature of the EMSTAC — LA relationships, and the 
LA activities that occur at the local level. 

After systematically analyzing the descriptive data fi'om LA interviews, EMSTAC has found 
that the patterns of EMSTAC support within Strategies I and II are fi-equently quite similar. Strategy 
I sites are currently in the fourth year of implementation with EMSTAC and Strategy II sites are in 
their third. These patterns seem to center around key EMSTAC predictor and outcome variables 
relevant to the development of relationships between EMSTAC personnel and LAs. These variables 
include; 

• Visits: The number of site visits made to a district. 

• Contact: The mean number of times per month that TA Liaisons and LAs communicate. 

• Support: The content of the majority of contacts between LAs and TA Liaisons. 

• Impact: The perception of how participation in EMSTAC has enabled the school/district 
to engage in activities that result in the implementation of research-based practices. 

Visits and Contacts 

During the first year working with EMSTAC, districts were in contact with TA Liaisons 
slightly more than twice per month (mean = 2.28 times per month). During that same initial period, 
TA Liaisons visited districts an average of 1.42 times per year (see Exhibit III- 17 below). 
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EXHIBIT 111-17: 
District Visits and Contacts 





Mean # of Monthly Contacts 
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Yr3 


Yr4 


Yrl 


Yr2 


Yr3 


Yr4 


Strategy I 


2.50 
n = 5 


2.20 
n = 6 


1.10 
n = 6 


.42 

n=6 


2.50 
n = 5 


1.50 
n = 6 


0.8 
n = 6 


0.67 
n = 6 


Strategy II 


2.30 
n = 10 


1.93 
n = 7 


1.65 

n=9 


N/A 


1.10 
n= 10 


1.07 
n = 7 


.56 
n = 9 


N/A 


Strategy HI 


2.05 

n=16 


N/A 


N/A 


N/A 


.66 

n=16 


N/A 


N/A 


N/A 


Average 


2.28 


2.10 


1.37 


.42 


1.42 


1.30 


0.68 


.67 



The frequency of contacts between TA Liaisons and LAs decreased over time. During the 
second year, districts and TA Liaisons were in contact an average of twice per month. The average 
number of visits fell slightly from the first year to the second year, from a mean of 1 .42 to a mean of 
1 .30 visits per year. In both of these instances Strategy III districts are included in the mean for year 
one, but not for years thereafter. Still, even without Strategy III districts the average number of visits 
are decreasing across Strategy I and II districts. 

This pattern applies across all the years in which Strategy I has been implemented. In the 
second year of working with EMSTAC, Strategy I districts had fewer contacts with EMSTAC staff 
than in year one, and this pattern continues through year four. The average number of Strategy I 
visits also decreases each year from year one through year four. This pattern is also emerging with 
Strategy II districts, whose LAs had fewer contacts with EMSTAC staff each succeeding year, and 
this pattern holds for the average number of visits. 

Descriptive data support the notion that most Strategy I and Strategy II LAs are reducing the 
number of contacts as their need for guidance from the TA Liaison diminishes over time. As LAs 
become more proficient with the implementation process, they use EMSTAC and the TA Liaison 
resources less and less. Most of the Strategy I LAs are using their skills and experiences to work on 
scaling up programs, either by concentrating on additional needs in the district, or by implementing 
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the intervention in other classrooms and schools to continue addressing the original need. Most 
Strategy II LAs report to be continuing with implementation of the original interventions, and are 
working on plans to scale-up current interventions or select new needs to address. 

Most LAs and TA Liaisons commented on the reduction of contacts and decrease in need for 
communication. In general, the increasing level of LA independence and autonomy seems to be 
related to the LA’s confidence and capacity to perform TA activities within the district, such as 
selecting and implementing research-based solutions for the district. 

As expected, information on contacts and average number of visits for Strategy III districts 
during year one shows they generally have less communication with their TA Liaison than LAs in 
Strategies I and II. This is a reflection of the differing structural designs across these strategies. 

The Nature of Contacts and Interactions Between EMSTAC and LAs 

After looking at the number of visits and contacts between TA Liaisons and LAs, EMSTAC 
investigated the various kinds of contacts. The results of our quantitative analysis illustrate that 
during the first year of working with EMSTAC, the common form of contact between LAs and TA 
Liaisons across the three strategies was district updates (46 percent). Frequent contacts were also 
focused on requests for information or research (19 percent). Another 12 percent communicated 
mainly about administrative issues. 

The pattern that is emerging so far is that LAs feel EMSTAC is very supportive and the TA 
support helps the districts with the entire change process, particularly with gathering research related 
materials and resources for intervention selection and implementation. The access to researchers and 
resources is consistently mentioned by Strategy I LAs, and these same LAs mention that EMSTAC’s 
financial resources are critical. TA Liaisons also seem to serve as motivators and facilitators of the 
process. Most Strategy I LAs did not access the website or the listserv as key components of 
EMSTAC’s support to them. 
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When exploring the data by Strategy, it appears that Strategy I districts were most likely to 
use their first-year of contact with EMSTAC for administrative issues (57 percent) and requests for 
information (29 percent). Conversely, Strategy II districts were more likely to use contact with 
EMSTAC for providing updates (56 percent), and requests for information (33 percent). Updates 
were the most common form of contact with EMSTAC for Strategy III districts (50 percent), with 
another 19 percent used for requesting information. 

Over time, the type of support provided by TA Liaisons to LAs did not appear to change 
dramatically. By the second year of working with EMSTAC, over half the districts still used 
opportunities to communicate with EMSTAC mainly to provide updates to TA Liaisons. The 
proportion of districts using TA Liaisons mainly for requests for information and research increased 
slightly over time. In the second year, one district used EMSTAC mainly for problem identification, 
while one other district used EMSTAC mainly to discuss administrative issues. 

By the second year, the majority of Strategy I district contacts with EMSTAC consisted of 
providing updates (57 percent). While requests for information was prevalent for Strategy II districts 
during Year 2 (50 percent), updates accounted for another 33 percent. Although updates were 
frequent for Strategy I and II districts during the second year of support, it is important to note that 
fewer Strategy I districts were dealing with administrative issues than in their first year, and more 
Strategy II districts were requesting research information than in Year 1 . This may have to do with 
the nature of training for Strategy II districts, as they were recruited, selected, and trained across a 
longer period of time than Strategy I sites. Thus, the majority of implementation activities for 
Strategy II sites occurred in the second year rather than in the first, which may have led to the desire 
for more access to information that could be disseminated to participating teachers and staff 

Finally, updates continued to be the most common form of contact with Strategy I districts 
during Years 3 and 4. Strategy II districts during Year 3 continued to focus mostly on requests for 
information and updates. During the early years of the project updates were used an as important 
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opportunity to encourage LAs with emotional support. As districts gained more confidence and 
capacity updates assumed a different nature, focused more on sharing and ongoing problem-solving. 
Still, updates continued to serve as an opportunity for experienced LAs to get recharged emotionally 
in their challenging role as agents of change. This type of support, combined with expert guidance 
and research-based information served as a core focus of the TA-LA exchange. 

Results of EMSTAC Supports and Relationships with LAs 

In the preceding section, the nature of the relationships between LAs and EMSTAC through 
number of contacts, visits, and the supports that LAs used (defined as the types of contacts that 
EMSTAC and LAs had with each other) was examined. It is also possible to examine the impact of 
these relationships in building local capacity to move research to practice by looking at LA interview 
data from participating school districts. This form of impact is defined as the “perception as to how 
participation in EMSTAC has enabled the school or district to engage in activities that result in the 
implementation of research-based practices.” This is measured by reviewing interview data to see 
how LAs perceive EMSTAC ’s effectiveness across various dimensions of support, including 
problem identification, identifying research-based materials, assisting with the coordination of TA 
events, facilitating support and follow-up, and motivating the LA. 

Several impact themes or dimensions emerge from the data, and they encompass all of the 
support variable dimensions listed above. The emerging impact dimensions are as follows: 

• Training 

• Resources: Financial, research-based materials and information, providing researchers 
and consultants 

• Connecting: Networking, referring to researchers and consultants 

• Process: Needs assessment, problem identification, problem solving, updates, 
administrative, evaluation 

• Motivation: Follow Up 
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While each of these dimensions impact LA capacity to move research to practice, there is evidence to 
also suggest their interaction is critical. This interaction occurs in similar and different ways across 
the three strategies. 

Strategy I 

In general. Strategy I LAs reported that the EMSTAC support they accessed throughout their 
participation with the Center had a high level of impact on their activities at the local level. 

According to LAs and other key players in the districts, EMSTAC affected these districts primarily 
through increased access to resources, with financial support being highly notable. The funds that 
EMSTAC provided to Strategy I districts served to support the LA and his or her efforts to facilitate 
change with teachers and other school and district based staff, and ultimately improve outcomes for 
students with disabilities in their districts. 

LAs also frequently described access to researchers and research-based materials as an 
important means through which EMSTAC developed both their knowledge base and confidence to 
move forward with their initiatives. Within Strategy I, there is clear evidence to suggest that 
EMSTAC’s impact is strong regarding the increased use of or access to research-based materials, 
researchers and consultants, and information regarding interventions to address the selected needs. 
LAs also reported that the various types of contacts, communications, visits, and on-going follow-up 
and updates served as motivational support and assistance throughout the process of change. 

Also notable is the less frequent use of EMSTAC’s technology resources by LAs in Strategy 
I, such as the listserv (“LA Exchange”), and the website. These resources provide access to on-line 
references, products, and links to research-based materials. It appears from LA comments that very 
few Strategy I LAs used the web-based resources or the listserv as a part of their support network; 
this could be explained in many ways, including the lack of those resources at the beginning of their 
involvement with EMSTAC. 
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District, as labeled by EMSTAC for confidentiality purposes, has been working with 
EMSTAC as a Strategy I district for the past four years, beginning in 1998-99, and their example 
provides a close up view of how EMSTAC supports Strategy I districts. As a result of EMSTAC’s 
work with the LA on their needs assessment results and other issues, the district implemented a 
number of research-based strategies from the Early Literacy Program (ELP) to improve the language 
and writing skills of students with disabilities in the primary grades. During the 1999-2000 school 
year, the LA began working with the TA Liaison to implement a new initiative, the SIMS program, a 
strategy instruction program designed to help students with study skills, reading comprehension, and 
self-monitoring skills. 

Initial training through the TA Liaison, followed by ongoing provision of information and 
assistance, gave the LA the confidence and capacity to spend considerable time with teachers and 
administrators, planning training activities, modeling techniques, coaching, and providing general 
support. During the past two years, however, the need for support and coaching has dwindled, and 
during the 2000-2001 school year, the LA narrowed her focus to scaling up programs to additional 
schools. The LA has primarily utilized EMSTAC through communication with the TA Liaison, 
participation in chat events and the listserv, and accessing information presented on both the public 
and private side of the website. Reflecting on her experiences with EMSTAC, the LA said EMSTAC 
services and supports were most valuable during the first two years of her work as an LA, and over 
time she gained a solid understanding of her role as a change agent and felt more confident in 
functioning in this capacity. She says the training and support received from her TA Liaison in the 
first and second year of the project helped establish a foundation for her future endeavors as a change 
agent. 

This example identifies a common thread cutting across several Strategy I sites; the 
importance of interpersonal relations, combined with TA capacity to link the LA with useful 
information relevant to local needs, on a timely basis. Strategy I fosters this capacity. Another 
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pattern found in this example is the capacity building that occurs. District reliance on EMSTAC 
decreases as EMSTAC builds their capacity to initiate and mobilize change. 



Strategy II 

Despite the difference between EMSTAC’s monetary support with Strategy I and Strategy II 
school districts, there is evidence that EMSTAC similarly impacted LA activities in Strategy II sites. 
For example, most Strategy II LAs, like Strategy I LAs, reported that the access to research-based 
programs, information, and materials was the most valuable support provided by EMSTAC. They 
also reported that involvement with the EMSTAC TA Liaison kept them motivated to persevere and 
stay the course with their initiatives. 

Strategy II LAs as a group are accessing electronic resources slightly more often than 
Strategy I LAs. This is to be expected in the aggregate considering there are more LAs in this cohort, 
though individual Strategy II LAs also appear to be accessing these resources more frequently. 

From both perspectives Strategy III LAs are accessing electronic supports more fi-equently than 
those in Strategies I and II. There are many potential explanations for the difference in the use of the 
technological resources, one of which is that Strategy I LAs were not explicitly trained to access 
these resources, since the private side of the website did not exist when Strategy I LAs were trained. 
Many of these resources were not ready for frequent use when some Strategy II LAs were trained as 
well. In contrast, the use of the web-based technology is thought of as the nexus of EMSTAC support 
for Strategy III sites, and was initially reserved for those districts. Lastly, LAs have inconsistent and 
unreliable access to the Internet, electronic mail systems, and the like. This creates a problem when 
trying to find a consistent and common means of communication with districts. 

A promising example of a Strategy II district benefiting from EMSTAC support is District 
12 as labeled by EMSTAC for confidentiality purposes. Since EMSTAC has been working with 
District 12, they have focused on a variety of initiatives, including contending with a citation from 
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the state for unusually high referral and placement rates of students in special education, improving 
reading and other literacy skills in students, increasing professional development opportunities for 
teachers, improving test scores of special education students, and providing services in the least 
restrictive environment. The LA and District 12 have found great benefit from working with 
EMSTAC on the Balanced Literacy and SAS initiatives. In this regard, the LA reports that 
EMSTAC has been particularly helpful in assisting the district to become more data-driven as it 
evaluates what works. The TA Liaison has supported the LA with these initiatives by listening to 
their needs, providing input, and solving problems when critical issues arise. The TA Liaison 
provides feedback on the LA’s planned activities (e.g., how to design and collect data to evaluate the 
intervention) and helps connect the LA with important resources. 

This example reinforces the common sense notion of what works in TA delivery: listening in 
a two-way exchange, and having the capacity to respond to needs in a timely fashion by knowing 
your client and connecting them to what works. This involves important interpersonal connections 
and critical knowledge and skills among the TA providers. Similar to Strategy I, by design Strategy 
II fosters these interpersonal connections. Typically, Strategy II LAs combine this with greater use 
of EMSTAC electronic resources than found among Strategy I LAs. 

Strategy III 

Overall, Strategy III LAs are positive about the support they are receiving from EMSTAC. 
Across the districts, access to research experts and information is seen as most helpful. Some 
Strategy III districts are especially positive about EMSTAC’s role in helping them move research to 
practice. For example, the LA in District 17, as labeled by EMSTAC for confidentiality purposes, 
offered the following comments on the progress they have made with their recent disproportionality 
initiative, “I just don’t think things could have progressed to the point where they are without 
EMSTAC.” Despite these mostly positive reports. Strategy III districts appear to be more likely to 
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report need for greater follow-up and communication with EMSTAC than their counterparts in 
Strategies I and II. 

Similar to the LAs in Strategies I and II, most Strategy III LAs reported in their interviews 
that they believe the most important support provided by EMSTAC is access to research-based 
materials and information, and that TA Liaisons are an important source of encouragement and 
motivation. Again, the relationship between EMSTAC and District 17 is a positive model from 
which we can gain understanding about this important activity. District 17 has been working with 
EMSTAC on the issue of disproportionate representation of minority students since 1999-2000. To 
address this problem, the district developed in collaboration with EMSTAC a preventive approach 
for reducing referrals for special education placement. The district's approach consisted of a 
comprehensive program that included curriculum based assessment (CBA) to conduct formative 
assessment and monitor progress in the acquisition of basic reading and literacy skills for students in 
danger of failing in reading. Prior to launching this initiative, the district engaged in a study 
examining the procedures for referral to special education and the pre-referral strategies utilized by 
the district to prevent inappropriate referrals. 

In implementing this initiative the LA facilitated consensus-building activities among 
teachers, administrators and other related school staff. She planned information and problem- 
solving meetings, and was the catalyst to ensure people were participating in the initiative. Much of 
this activity has been based on research information provided by EMSTAC. The LA has been 
connected with research materials, experts, and video presentations. 

The LA indicated that support received through her EMSTAC TA Liaison was crucial to 
their progress at every step. Primarily, the LA utilized the TA Liaison for gathering and collecting 
research about disproportionality in order to build the LA’s knowledge base on the issue. The LA 
also communicated regularly with the TA Liaison for the purposes of touching base and updating the 
TA Liaison on new progress and activities. In turn, the TA Liaison provided strong encouragement 
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and momentum for the LA to continue with their initiative. The LA said “the connection that 
EMSTAC provides between theory and practice is very good and very solid. I’ve taken things to 
people in other districts because of our collaboration with EMSTAC. We are now the model in the 
state for looking at disproportionality.” 

One of the critical features of this case is the close relationship that developed between the 
TA Liaison and the LA, one characterized by trust, ongoing encouragement, and a two-way exchange 
where co-leaming occurs. This created the conditions enabling the TA Liaison to respond to the 
district’s needs in a timely and effective manner. This was further enhanced by the expertise of the 
TA Liaison and the usefulness of information provided to the LA. 

This situation focuses attention on the power of interpersonal relations to make the difference 
in TA assistance. From our perspective, what is also noteworthy is the fact that this occurred in a 
Strategy III district, though it is not representative of all TA relationships functioning under this 
structure. Strategy III does not involve the personal hands-on training provided by TA Liaisons in 
Strategies I and II, and involves fewer structured visits and contacts. Consequently, it is not 
surprising that a larger number of Strategy III LAs report that they would benefit by more 
communication and follow-up with their TA Liaison. What occurred in District 17 can happen under 
any of the three strategies where individuals with the right mixture of knowledge, skills, and 
interpersonal rapport come together. Still, Strategy III as designed does not structure and facilitate 
this as do Strategies I and II, and this is reflected in the more uneven experience of Strategy III 
districts. 

As cited in our description of EMSTAC supports, this past year the majority of participants in 
the EMSTAC electronic supports were Strategy III LAs . While Strategy III relies less on 
interpersonal relations than the other strategies, the emphasis on distance education and electronic 
support tools during our Strategy III recruitment work may have either attracted LAs who are more 
likely to use these resources, or may have developed their confidence and interest in taking 
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advantage of these resources. Despite this pattern, a few Strategy III LAs are not accessing these 
supports. These LAs report feelings of disconnection from the project, which resonates with the 
notion of needing more frequent interpersonal contact with EMSTAC. One LA said he needs a 
regular prompt to keep him alert to what is available through the EMSTAC electronic supports. 

Relationships Between LAs and Teachers 

As with the relationships between EMSTAC and LAs, it is difficult to generalize about the 
nature of relationships between local LAs and the school and district staff with whom they work to 
implement research-based practices. Each district is unique and each LA reflects several of the 
unique and complex aspects of school districts, regardless of their position within the district. Each 
LA works in a different district or school-based capacity, has distinct goals, and has various roles and 
responsibilities. However, the descriptive evidence collected from teachers does suggest patterns in 
the nature of LA relationships with teachers and staff, and in the degree to which LAs have an impact 
on teachers. 

The local impact score is determined by the average rankings teachers report regarding the 
various supports the LA provided throughout the change process, taking into consideration any 
comments or thoughts of administrators and other people working closely with the LA. Teacher data 
were collected via a survey instrument, which included both Likert scale and open-ended questions. 
When considering these data, we examined the nature of the LA-staff relationships and the ways in 
which the reported impact on teachers has occurred and through what kinds of supports. 

Results were compiled by analyzing data from each teacher survey item. Items 18-28 on the 
Teacher Survey were most relevant to this outcome variable, so EMSTAC selected the top five 
responses from these Likert scale items and the top three emerging themes from the open-ended 
questions (items 29 & 30) for each district, in order to inform the development of common themes in 
LA support. The items with the top five scores and data most relevant to this variable were 
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considered the most influential aspects of the LA’s support to the district or school. This score only 
tells us something about the nature of those activities if we disaggregate the data and look at the 
individual scores for each item. 

The results of the Likert scale item analysis show that teachers feel the programs they are 
implementing and their work with the LA has helped them learn new techniques and change how 
they think about teaching students with special needs. They also reported that they believe the 
programs help them respond to student needs, that students enjoy the interventions they are using, 
and that the interventions have had some positive impact on student achievement. The following 
were the highest ranked Likert Scale items in the Outcomes section of the survey: 

• Question 18: 1 have learned new techniques to teach my students as a result of my 
involvement with the program. 

• Question 19: This program has had some influence on how 1 think about teaching 
students with special needs. 

• Question 20: This program has increased my awareness of the learning needs of students 
with disabilities. 

• Question 21:1 believe that the program is effective in making instruction more 
responsive to student needs. 

• Question 25: Students seem to enjoy the learning activities associated with this program. 

• Question 26: 1 believe that the implementation of this program has had a positive impact 
on student achievement. 

• Question 28: 1 see myself using these techniques five years from now. 

Results from the analysis of the open-ended questions show that teachers felt supported most by LA 
activities that 1) create a smooth transition for the use of the intervention/s, 2) help maintain on-going 
implementation of the interventions (offering techniques, feedback on performance, and follow-up 
through visits and consistent meeting times), and 3) offer personal contact and support (such as 
encouragement, personal attention, and consistent one-on-one communication). We can 
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conceptualize these into different dimensions, similar to those created for the various influences of 
EMSTAC’s support to LAs. These are the following support dimensions: 

• Transition and Training 

• Implementation 

• Motivation 

• Resources 

• Process Facilitation 

Although not all of the LAs conducted all of these activities for their districts, most LAs engage in 
each of these activities to some degree. Collected data further show that teachers believed the 
interventions they were using 1) increased their awareness of the needs of students with disabilities 
and that the programs, 2) were helpful in creating instructional environments more conducive to the 
needs of their students, and to a lesser degree, 3) the programs did have some influence on how they 
view teaching students with special needs, and 4) that it seems the students enjoy the interventions. 

In summary, comparing the impact of strategies demonstrates that EMSTAC has effectively 
trained local LAs to: 

• Identify and support implementation of research-based practices, 

• Positively impact systemic change efforts at the local level, 

• Scale-up research based practices to other schools within the district, and 

• Solve new problems using research-based solutions. 

EMSTAC’s training and support have produced sustained impact and an important capacity for 
problem solving among participating school districts. 

Child Outcome Data 

Thus far we have discussed impact in relation to the process of moving research to practice. 
Such practices have been scientifically validated as producing results for children with disabilities 
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through other projects and studies. However, we discuss below, examples of child outcome data 
obtained from a representative sample of 14 EMSTAC school districts. Each of these are examples 
where EMSTAC helped the district implement an intervention that improved outcomes. We have 
organized these data by district within four categories: (a) behavior/school-wide discipline, (b) 
reading and other areas of academic instruction, (c) inclusion/accessing the general education 
curriculum, and (d) disproportionate representation of minority students in special education 
programs. These categories reflect the four main topical areas for technical assistance that have been 
requested to date by EMSTAC’s school district partners. 

Behavior/ School-wide Discipline 

District 2. EMSTAC’s work with District 2 resulted in the implementation of Project 
ACHIEVE, a school-wide model that includes a specific focus on addressing discipline concerns. 
Implementation of Project ACHIEVE began at a middle school in this district in spring of 1999. 
Although minimal impact was seen in office referrals during the first year of implementation, by the 
2000-2001 school year a dramatic decrease in all types of code of conduct violations was 
documented. For example. Class 1 offenses (classroom disruption, insubordination, verbal abuse) 
dropped from 1,914 referrals to 931. Class 2 offenses, which include fighting, vandalism, and 
stealing, dropped 42 percent from 394 referrals to 227 referrals. Finally, the most serious form of 
violations including battery, possession of a weapon and arson, dropped from 18 incidents in 1999- 
2000, to eight incidents the following year. 

District 15. In this school district, district personnel established an alternative program to 
serve students who exhibited violent or severely inappropriate behaviors and could not remain in a 
general classroom or who had been previously expelled due to behavior infractions. The aim of the 
program was to ensure academic instruction by also focusing on research-based practices such as 
positive reinforcement and social skills training in general education to promote reintegration. 
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Additionally, families attended parenting classes. Students remained in the alternative program for a 
minimum of two months. Seven of the eight participating students have been fully integrated into 
their first and second grade classrooms. (The eighth student was placed in a psychiatric program in a 
local hospital.) Results from a survey of 30 teachers indicated that 90 percent were satisfied with the 
program and wanted to see it continued. 

Reading/ Academic Instruction 

District 1. This district implemented a number of research-based strategies from the Early 
Literacy Program (ELP) in order to improve the language and writing skills of students with 
disabilities in the primary grades. ELP is a research-based literacy program developed by Carol Sue 
Englert and others at Michigan State University. The data reported here represent Mean Percentiles 
for one cohort of students on the Comprehensive Test of Basic Skills (CTBS). The data reported 
indicate student performance on the CTBS both before (1998) and after (2000) implementation of 
ELP. As indicated in the scores below, the performance of this cohort of students increased 
significantly. 





1998 


2000 


Mean Percentiles 






Language 


43.0 


59.5 


Lang. Mechanics 


43.0 


54.0 


Lang. Composite 


47.0 


57.0 



District 24. In the 1999-2001 school years, a middle school in this district began 
implementing a school-wide, research-based literacy program that included literacy coaches at each 
of the schools, remedial reading classes, and an emphasis on reading and writing throughout the 
curriculum. New instructional materials were selected and acquired to enable staff to more easily 
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implement this literacy initiative. Additionally, during book club meetings staff taught targeted 
students research-based strategies to improve their reading comprehension. Lastly, the school 
utilized students’ standardized testing results to drive future instructional practices. After three years 
of implementation, the percentage of students who met state proficiency standards has increased, 
based on state assessments, in 1) reading from 70 percent in 1999 to 75 percent in 2001 and 2) math 
from 50 percent in 2000 to 66 percent in 2001 (math proficiency test was not implemented until 
2000). Meanwhile, the percentage of students proficient in writing has remained relatively stable (48 
percent in 1999, 47 percent in 2001). 

Districts. Elementary schools in this District initiated the implementation of Direct 
Instruction and other research-based strategies to teach phonemic awareness skills to students with 
disabilities in kindergarten and first grade. Data collected during the initial year of the project 
documented the impact of these strategies on students’ early reading skills. For example, in one 
school building, letter naming fluency increased from 32 letters per minute to 54 letters per minute 
during the first six months of program implementation for first grade students. This increase 
represented a 68 percent improvement in letter naming fluency. Further, during this same time 
frame, student ability to correctly segment words into syllable parts increased 76 percent. 

District 7. At one elementary school in this District, Peer Assisted Learning Strategies 
(PALS) was implemented to increase active engagement and learning, reading fluency, and reading 
comprehension among students with disabilities. Curriculum Based Measurement (CBM) was used 
to track student progress school-wide. Results from the CBM probes documented student growth in 
the number of words read correctly per minute (WCPM), between the fall and following spring 
during the initial year in which PALS was implemented. While gains were documented at all levels, 
second grade classrooms, which increased their WCPM scores between 40 percent and 97 percent, 
demonstrated the highest percent increase. 
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District 3S^. In one school district, EMSTAC assisted in initiating implementation of a 
phonemic awareness program for children with disabilities in kindergarten and first grade. Using the 
Dynamic Indicators of Basic Early Literacy Skills (DIBELS), one school has been able to document 
growth in the acquisition of early reading skills by each new kindergarten cohort that is exposed to 
instruction in phonemic awareness. As a result of three years of exposing incoming kindergarten 
students to the program, the percentage of kindergarten students demonstrating sustained skills in 
phonemic segmentation fluency has increased from 21 percent to 39 percent. 



Inclusion/Accessing the General Education Curriculum 

District 13. Schools in this district have implemented an Inclusion Initiative based on 
research- based methodologies adapted from the Emory Autism Program and Project Winning Team. 
District 1 3 maintained a continuum of special education services while continually reevaluating 
students to ensure that they were in the least restrictive environment with maximum exposure to the 
general education curriculum. In 16 of the 21 schools, the Inclusion Initiative has been implemented 
and will be in the remaining five schools during the 2001-2002 academic year. Success of the 
program has been measured by the academic gains made by students in special education. For 
instance, 92 percent of students with disabilities enrolled in general education classes passed the 
math and reading sections on a state achievement test. Additionally, teachers have been able to 
identify and cultivate strengths of students in special education; evidence of this finding is the fact 
that 30 special education students have been referred and determined to be eligible for gifted 
services. 

District 5. This District recognized the need to provide students with severe disabilities 
increased access to meaningful inclusion experiences. As a result, the district targeted one school to 
implement a collaborative model in which parents, special education teachers, general education 

* District 33 provided student outcome data but was not included in the sample of districts in the 2002 evaluation. 
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teachers, and trained paraprofessionals worked together to include targeted students in general 
education classrooms. Prior to the 1998 school year, no students with severe disabilities were 
included in general education in the targeted elementary school. During the first year of working with 
EMSTAC, one student with Down’s Syndrome was included and successfully supported. Since then 
the number of students with severe disabilities receiving services within the general education 
environment has increased to nine. Reports from the school indicate that these students have made 
significant progress in attaining their lEP goals. 

District 3^. To increase the number of students with disabilities being educated in general 
education classrooms. District 34 adopted a co-teaching model in which a general education teacher 
and a special education teacher collaborate to teach a diverse group of learners in one classroom. 
General education teachers are provided in-service workshops to learn about inclusion and the 
process of collaboratively adapting and modifying curriculum for students with special needs. To 
support the inclusion model, the district provides additional staff in inclusive classrooms and 
provides teachers in these classrooms with information on validated, research-based strategies for 
effectively instructing students with disabilities in general education. After three years of 
implementing these programs, the percentage of students with disabilities being educated in general 
education classrooms has increased from 26 percent to 40 percent. Survey results of general 
education teachers in the district show that about one-third of the teachers report a positive 
experience with inclusion-based practices, while two-thirds report some positive experiences and 
some negative experiences. None of the teachers participating in the survey indicated only negative 
experiences working with the co-teaching model to support inclusion of students with disabilities. 



’ District 34 provided student outcome data but was not included in the sample of districts in the 2002 evaluation. 
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Disproportionate Representation 

District 17. To address the overrepresentation of minority students in special education, the 
District in collaboration with EMSTAC developed a preventive approach for reducing referrals for 
special education placement. The district's approach consisted of a comprehensive program that 
included curriculum based assessment (CBA) to conduct formative assessment and monitored 
students' progress in the acquisition of basic reading and literacy skills for students in danger of 
failing in reading. CBA data revealed an improvement in reading fluency rates for both passages and 
words in isolation. For example, there was an average increase of 1 1 . 1 words for the number of 
words read correctly in a passage and an average increase of 1 1.8 words for the number of words 
read correctly from a list of high frequency, sight vocabulary words. 

District 8. This District implemented insfructional support teams (ISTs) in 17 schools for 
students in general education classes who were at-risk for referral for evaluation and possible 
placement in special education. The 1ST model promoted the success of students by meeting their 
needs (educational, behavioral, and social) in the general education classroom. After implementing 
the program from 1998-2001, the number of students receiving special education services 
represented less than one percent of the total enrollment in the 1 7 1ST schools over the three-year 
period. Further, seven schools have improved their composite index scores on a district wide 
assessment for grades 3 and 5, respectively, since the inception of the 1ST program. 

District 22. District 22 implemented a comprehensive program that included research-based 
academic and behavioral interventions for minority students at risk for special education placement. 
The program components were designed to improve students' documented behavior and reading 
problems. Data from the district-wide assessment revealed an improvement for the percentages of 
African American and Hispanic students, in third grade, scoring at or above grade level. For 
example, across two school years (from the 1999-2000 to the 2000-2001 school years), the 
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percentages of African American students who met or exceeded grade level expectations in reading 
increased from 72.3 percent to 75.1 percent. In addition, the percentages of Hispanic students who 
met or exceeded grade level expectations in reading increased from 77.3 percent to 79.7 percent. 

District 30. One middle school in this District implemented a comprehensive research- 
based program for minority students at risk for special education placement. The program included 
mentoring, a Saturday tutorial program, technology integration and a reading comprehension 
program. The program components were designed to address students' documented behavior and 
reading problems. Data from a state assessment of academic skills revealed an increase from 125 
students passing in the spring of 2000 to 141 students passing in the spring of 2001. 
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STATE AND STRATEGIC 

PARTNERSHIPS 



State and Strategic Partnerships 



From early on in its operation as a national technical assistance center, EMSTAC has 
recognized the importance of moving beyond an individualized district-by-district approach to 
recruitment, training, and TA support. The inherent limitations in attempting a technical assistance 
partnership with the more than 15,000 local school districts across the country suggests the 
development of additional recruitment, training, and follow-up TA support strategies to successfully 
meet the goal of providing meaningful resources to schools. With this challenge in mind, EMSTAC 
evolved beyond its initial district-by-district recruitment and training approach to establish strategic 
partnerships at multiple levels, including forming alliances at the SEA level and with special 
education providers operating at regional and LEA levels. Several important outcomes related to the 
use of multiple level recruitment, training, and support strategies have emerged: 

• EMSTAC has attempted to match its training and product resources with state and 
regional-level technical assistance priorities and professional development goals; 

• By partnering with SEAs and other relevant organizations, EMSTAC can access a larger 
number of local school districts within a state than by individual LEA recruitment efforts 
alone; 

• EMSTAC has maximized access to potential LEA partners on a state, regional and local 
level by strategically partnering with other organizations with established LEA networks. 

To meet these goals EMSTAC has established working relationships with the following state 
offices of special education: Alabama, Alaska, Arizona, Connecticut, District of Columbia, Iowa, 
Kansas, Louisiana, Maryland, Michigan, Montana, New Mexico, New York, North Carolina, South 
Carolina, South Dakota, Utah, and Wyoming. Exhibit III- 18 lists the state partnerships that EMSTAC 
has currently established. One of the preliminary lessons learned from this work is that one size does 
not fit all: there is significant variability across states in the match between state-level technical 
assistance priorities and those held by local districts. EMSTAC has supported SEA technical 
assistance priorities while also maintaining sufficient flexibility for effectively addressing local 
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district needs. This approach has allowed EMSTAC to have maximum efficiency in accessing a 
greater number of districts while also working within a structure that provides resources that are most 
relevant to individual district goals. 



EXHIBIT 111-18: 
State Partnerships 



State Partnerships 


Year Established 


Alabama 


Year 4 


Alaska 


Year 4 


Arizona 


Year 4 


Cormecticut 


Year 4 


District of Columbia 


Year 2 


Iowa 


Year 4 


Kansas 


Years 


Louisiana 


Year 2 


Maryland 


Year 1 


Michigan 


Year 1 


Montana 


Year 4 


North Carolina 


Year 4 


New Mexico 


Year 4 


New York 


Year 4 


South Carolina 


Year 2 


South Dakota 


Years 


Utah 


Years 


Wyoming 


Years 



In addition to supporting SEAs, EMSTAC has also developed strategic partnerships with 
other educational organizations. Exhibit III- 19 provides an overview of the strategic partnerships that 
EMSTAC has established. These organizations include the Council of Administrators of Special 
Education (CASE); Council of Chief State School Officers (CCSSO); Eugene Research Institute; 
Mountain Plains Regional Resource Center (MPRRC); 100 Black Men of America, Inc.; Parents Inc. 
(Alaska); Phillips 66 (Alaska); University of Montana; Northeast Regional Resource Center 
(NERRC); and Southeast Regional Resource Center (SERRC). The remainder of this section 
provides an overview of recruitment activities to date with a sample of state and strategic partners 
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currently working with EMSTAC. As an outcome to this effort 16 LEAs from these states and 



partner organizations have joined the EMSTAC roster of district partners. 

EXHIBIT 111-19: 
Strategic Partnerships 



Partnership Organization 


Year Established 


Council of Administrators of Special Education 


Year 1 


Council of Chief State School Officers 


Year 1 


Eugene Research Institute 


Year 1 


Mountain Plains Regional Resource Center 


Year 1 


100 Black Men of America, Inc 


Year 4 


Parents, Inc. (Alaska) 


Year 4 


Phillips 66 (Alaska) 


Year 4 


University of Montana 


Year 4 


Northeast Regional Resource Center 


Years 


Southeast Regional Resource Center 


Years 



Council of Administrators of Special Education 

The partnership that EMSTAC has developed with CASE comes through the support of its 
former executive director, Jo Thomason, and ongoing district work with CASE consultant Jo Paroz. 
Both Dr. Thomason and Ms. Paroz have participated in EMSTAC recruitment, training activities at 
conference presentations and other events, and TA support. This effort has resulted in seven LEA 
partners who have completed training. 



100 Black Men of America, Inc. 

In May 2001 EMSTAC and the 100 Black Men of America, Inc., agreed to partner on an 
initiative for addressing minority disproportionality — the overrepresentation of children of color in 
special education. This joint endeavor centers on the provision of EMSTAC training and follow-up 
TA support to local school districts participating in the 100 Black Men of America’s Wimberly 
Initiative - student mentoring program. This strategic partnership was formed in order to help 
facilitate the delivery of effective school-based programs, teacher supports, and resources for 
minority students placed into special education and for those considered to be at risk for special 
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education referrals. Dr. Leroy Ervin, National Chairman of the 100 Black Men of America, Inc.’s 
Educational Policy Committee, is working with EMSTAC on this initiative. LAs have been trained 
in Charlotte-Mecklenburg, North Carolina; Las Vegas, Nevada; Memphis, Tennessee; and San 
Antonio, Texas. Representatives from these districts also participated in the EMSTAC pre-institute 
given at the May 2001 CCSSO conference on high poverty schools in Baltimore, Maryland. With 
over 10,000 members in 82 chapter cities, the 100 Black Men of America, Inc. is an important asset 
to the effort by EMSTAC to, on a national scale, provide training and other technical assistance 
resources to LEAs. 

Mountain Plains Regional Resource Center 

The MPRRC has provided the services of one of its program specialists, Mr. Wayne Ball, to 
work directly with EMSTAC on recruitment, training, and TA support for school districts from 
within its 10-state region, which also includes the Bureau of Indian Affairs. Mr. Ball has worked 
with EMSTAC staff across a variety of recruitment and training activities including national 
conference presentations, LA training events, and recruitment of individual school districts within the 
MPRRC network. The EMSTAC/MPRRC partnership has resulted in the effective recruitment and 
training of three LEAs. 

Parents, Inc. 

Parents, Inc. is the OSEP-fiinded Parent Training and Information Center for the state of 
Alaska. It provides training resources, support, and advocacy. Through EMSTAC’s partnership with 
Parents, Inc., LAs have been recruited and trained in two LEAS. 

Council of Chief State School Officers 

Through its Initiative to Improve Achievement in High Poverty Schools and its Title I/IDEA 
working group, the Council of Chief State School Officers (CCSSO) has access to local school 
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districts and state-level staff across 40 states. For this strategic partnership, EMSTAC worked with 
Elaine Bonner-Tompkins, CCSSO’s Senior Project Associate in Special Education, to integrate 
EMSTAC resources with the CCSSO network of schools. CCSSO supported EMSTAC recruitment 
activities by promoting EMSTAC in its newsletter, distributing recmitment letters, and facilitating 
EMSTAC participation in CCSSO conferences and training events for new LAs, including the May 
2001 High Poverty Schools conference in Baltimore, Maryland. At this conference EMSTAC 
coordinated a pre-conference institute and provided an overview of its LA training to conference 
attendees. Through the CCSSO partnership EMSTAC gained access to LEAs and state office staff 
from 14 states as well as the Bureau of Indian Affairs. 

As a result of the various recmiting methods employed by EMSTAC, to date over 3000 local 
districts across the country have received information about EMSTAC resources. From this base of 
recmited districts, 162 local districts have followed up with requests for additional information (e.g., 
the EMSTAC “Welcome Packet”). Sixty-four local districts have moved to the next step of 
designating an LA and beginning the EMSTAC training. Finally, 47 local districts to date have 
completed the training process and are full EMSTAC partners. The use of strategic partnerships was 
found to be the second most effective method of recmiting LEAs and LAs, second only to direct 
professional referrals. Of the 48 local districts expressing interest due to this recmitment method, 16 
completed training for a yield of 33 percent. 
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EMSTAC COSTS 



EMSTAC Costs 



The EMSTAC Cost Comparison section that follows includes a discussion of costs incurred 
by EMSTAC in the delivery of technical assistance to school districts across the three TA strategies. 
Consistent with past EMSTAC cost comparisons, this analysis will focus on the following three 
functional areas; 

♦ Recruitment of school districts and LAs 

♦ Training of LAs 

♦ Technical Assistance (TA) support to LAs and districts 

Procedures for Making Cost Comparisons 

In order to make cost comparisons, we used an estimation of percentage of costs that was 
incurred for each of the functional areas (recruitment, training, and TA) across the three TA 
strategies. This involves using the most accurate EMSTAC cost data available, with estimations used 
because of overlapping cost categories. TA Liaisons estimated the percentage of labor costs that 
were incurred in recruiting school districts and LAs, training LAs, and providing TA support to LAs 
and school districts per each of the three TA strategies. 



Problems Encountered 

As was discussed in the EMSTAC 2001 Evaluation Report and Adjustment^ , several 
problems were encountered related to availability and comparability of data, including: 

• Availability of Timely Cost Data: The analyses included here involve expenses 
(EMSTAC labor and other direct costs) incurred through May of 2002. As such, 
expenses for activities incurred since May are not part of this analysis. This is 
particularly important when we view the functional areas across time because the data for 
Year 5 is only for eight months, while the data for the previous years is for 12 months. 



Mesmer, E., Ritter, S., Paulsen, C., Carl, B., Dailey, D., Shami, M., Hamilton, J., Mclnemey, M., & Gerver, M. 
(2001). The Elementary and Middle Schools Technical Assistance Center: Evaluation report and adjustments from 
Year 4. 
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This naturally leads to lower costs for Year 5 because the previous years include four 
additional months of cost data. Furthermore, some costs incurred by EMSTAC are not 
included if billing has not been submitted by our partner organizations and 
subcontractors. 

■ Matching Staff Time with Three Areas of Technical Assistance: EMSTAC cost records 
are reasonably accurate in terms of expenses incurred for travel and other non-labor 
expenses, but are less attributable to how staff time was allocated across the three 
functional areas. It has been necessary to generate estimates of how staff time has been 
allocated. These estimates were made by having staff fill out brief surveys that asked 
them to allocate their time on a percentage basis to each function of interest (recruitment, 
training, and TA support). 

• Overlapping and “Nonspecific” Expenses: Perhaps the most significant problem in this 
comparison of costs was encountered in the overlap of costs. Simply put, many, if not 
most, of our expenses to date are not readily attributable to one particular function or 
strategy. The solution that was used to solve the problem of overlapping costs was to split 
them, wherever possible, across functions. For example, a TA Liaison may have attended 
a conference for Strategy III recruitment purposes and provided TA support by visiting a 
Strategy I LA within the same trip. Thus, it has been extremely difficult to calculate and 
compare costs and thus make definitive statements about the relationship of these costs to 
each TA strategy. As a result, the information that has been presented in this cost section 
of the report should be interpreted with some caution. The EMSTAC 2001 Evaluation 
Report and Adjustments^ discusses this issue in greater detail. 

• Allocation of Costs for Major EMSTAC Partner Organizations and Subcontractors: For 
EMSTAC’s major partner organizations and subcontractors, we estimated how their 
funding has been allocated across the three functional areas of interest for Year 5. Please 
see the EMSTAC 2001 Evaluation Report and Adjustments for allocations across the 
three functional areas for previous years. The estimations for year five are as follows: 

o Council of Administrators of Special Education (CASE): 

■ 60 percent Recruitment 

■ 40 percent TA Support 

o Council of Chief State School Officers (CCSSO): 

■ 100 percent Recruitment 

o Mountain Plains Regional Resource Center (MPRRC): 

■ 75 percent Recruitment 

■ 25 percent TA Support 



' Mesmer, E., Ritter, S., Paulsen, C., Carl, B., Dailey, D., Shami, M., Hamilton, J., Mclnemey, M., & Gerver, M. 
(2001). The Elementary and Middle Schools Technical Assistance Center: Evaluation report and adjustments from 
Year 4. 
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Eugene Research Institute (ERI): 

90 percent TA Support 
5 percent Recruitment 
5 percent Training 

o Parents, Inc.: 

■ 70 percent Recruitment 

■ 30 percent TA Support 

o 100 Black Men of America, Inc.: 

■ 50 percent Recruitment 

■ 50 percent TA Support 

Areas of Cost Not Included in This Analysis 

EMSTAC has incurred substantial expenses in areas that do not align with the three areas of 
interest in this section (recruitment, training, and TA support). Consistent with previous analyses, 
areas not included in the cost comparison include: 

❖ Travel costs for LAs (e.g., to attend conferences) 

* 1 * Consultants used by EMSTAC school districts (e.g., for staff development activities) 

* 1 * Reviewers used for EMSTAC products and Website 

❖ Meetings of EMSTAC’s Advisory Group 

*♦* Stipends paid to Strategy I and II districts (see discussion under TA Support costs section 
below) 

❖ Staff time for temporary help and staff who no longer work with EMSTAC 

❖ Data collection and analysis for EMSTAC Evaluation 

Comparison of Costs by Function and Strategy 

The following section of the analysis looks at percentage of costs associated with each of the 
three functional areas of interest (recruitment, training, and TA support), with relevant comparisons 
across strategies. As seen in Exhibit III-20, 62.7 percent of total expenses^ were spent on technical 



* Total expenses include cost of labor and non-labor. 
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assistance support to LAs and school districts. Consistent with the April 2001 analysis, TA support 
continues to be the functional area in which most costs are incurred. Since the previous analysis, 
however, there have been slight changes regarding the percentage of expenses spent on recruitment 
and training. Recruitment has increased, while training has decreased. Recruitment had accounted for 
16 percent of total expenses, it now accounts for 18.8 percent of total expenses. Training had 
previously accounted for 20.2 percent of total expenses, but has since dropped to 18.5 percent of total 
expenses. This may be attributed to Strategy Ill’s online training because the cost decreases as each 
new Strategy III district is trained. 

EXHIBIT 111-20: 

Share of Combined Total Expenses for Recruitment, Training, and TA 
Support, by Functional Area (Total = 100%) 












Recruitment 



Training 



TA Support 



A breakdown of labor expenses for each functional area (recruitment, training, and TA 
support) can be seen in Exhibit III-21. The current analysis finds the majority of labor expenses 
incurred are for TA support (71.3 percent). Labor costs for training and recruitment account for 15.5 
percent and 13.3 percent, respectively. The current analysis shows evidence that the final year of the 
project emphasized technical assistance support to LAs and school districts. Compared to last year’s 
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analysis, there was a decrease in the percentage of labor costs associated with recruitment and 
training and an increase in the percentage of labor costs for TA support. 



EXHIBIT 111-21: 

Percentage of EMSTAC Direct Labor Costs by Functional Area* 




*Data used from sample of nine TA Liaisons. 



Through the data reported in the TA Liaison Survey and EMSTAC cost records, labor cost 
data seem to be a more reliable estimation of costs than the non-labor costs. Exhibit III-22 shows the 
distribution of cost across years for each TA strategy. For each strategy, TA support has accounted 
for the majority of labor costs (Strategy I = 95.4 percent. Strategy II = 62.2 percent. Strategy III = 
70.6 percent). Labor costs associated with recmitment for Strategy I were not incurred by any of the 
nine TA Liaisons that were surveyed for labor cost data. For Strategy II, greater expense was 
incurred for labor costs to train LAs (21.5 percent) than to recruit school districts and LAs (16.4 
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percent). For Strategy III, however, greater expenses due to labor costs were incurred for recruiting 
school districts and LAs (15.7 percent) than for training LAs (13.7 percent). This may be attributed to 
the online nature of Strategy III training, which requires less time of EMSTAC staff 

EXHIBIT 111-22: 



Labor Cost for Functional Areas Per Strategy* 



Strategy 


Recruitment 


Training 


TA Support 


Total 


I 


— 


4.6% 


95.4% 


100% 


II 


16.4% 


21.5% 


62.2% 


100% 


III 


15.7% 


13.7% 


70.6% 


100% 



* Labor costs are based on the labor of a sample of nine TA Liaisons through May 2002. 



Exhibit III-23 shows the distribution of labor costs across time for each of the three TA 
strategies. Each of the TA strategies was phased in over time, thus the missing recruitment data for 
Strategy I would have taken place during Year 1. Strategy II recruitment and training began in Year 
2. Strategy III recruitment and training began in Year 4. For each of the TA strategies, it is typical for 
the labor expense to decrease for each function over time. This is partially attributable to EMSTAC’s 
five-year model that shifted fi’om recruitment and training of Strategy I school districts and LAs to 
recruitment and training of Strategy II LAs, then again to recruitment and training of Strategy III 
LAs. As we shifted to each new strategy, the labor costs for the previous strategy decreased because 
we were no longer recruiting for that strategy and the LAs had completed training. Regarding TA 
Support, it has been found that LAs typically require less intensive support over time. 

It is important to note that different functional areas were emphasized as new strategies were 
implemented. Recruitment was emphasized more in Strategies II and III because Strategy I was 
limited to seven school districts with pre-arranged contacts. Training differed across strategies, as 
will be discussed in the Training section. With the implementation of Strategy III, EMSTAC added 
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an additional functional area that is not represented in this section, but warrants mention. EMSTAC 
labor was used to support topical teams and product development once Strategy III was implemented. 
This has not been included as TA support because it is used to support all EMSTAC sites, rather than 
a specific school district or Linking Agent. 

EXHIBIT 111-23: 



Labor Cost for Functional Areas Across Time and Strategy* 



Year 


Recruitment 


Training 


TA Support 


Total 


1 


N/A 


N/A 


N/A 


N/A 


2 


SI; n/a 
S2: 28.5% 
S3: n/a 


SI: 4.9% 
S2: 18.4% 
S3: n/a 


SI: 31.6% 
S2: 16.6% 
S3: n/a 


100% 


3 


SI; n/a 
S2: 8.3% 
S3: N/A 


S 1 ; n/a 
S2; 14.8% 
S3: n/a 


SI: 28.2% 
S2: 48.7% 
S3: n/a 


100% 


4 


SI: n/a 
S2: .3% 
S3: 14.9% 


SI: n/a 
S2; 6.2% 
S3: 8.6% 


SI: 10.6% 
S2: 23.5% 
S3: 36.0% 


100% 


5 


SI: n/a 
S2: 2.6% 
S3: 1.9% 


SI: n/a 
S2: 4.0% 
S3: 8.2% 


SI: 7.2% 
S2: 22.0% 
S3: 54.0% 


100% 


All Years, Per 
Strategy 


SI: n/a 
S2: 7.1% 
S3: 6.2% 


SI: 0.8% 
S2: 9.3% 
S3: 5.4% 


SI: 16.6% 
S2; 26.9% 
S3: 27.8% 


100% 


All Years, 
Across 
Strategies 


All: 13.3% 


All: 15.5% 


ALL: 71.3% 


100% 



* Labor costs are based on the labor of a sample of nine TA Liaisons through May 2002. 

* SI = Strategy I, S2 = Strategy II, S3 = Strategy III 



Recruitment 

As seen in Exhibit III-20, 18.8 percent of the combined total expenses incurred by EMSTAC 
to date across the areas of recruitment, training, and TA support can be attributed to recruitment- 
related activities. The seven districts in Strategy I were recruited primarily through pre-existing 
contacts of EMSTAC staff The small number of these districts, combined with the pre-arranged 
contacts and working relationships that had been established, generally meant that recruitment costs 
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associated with Strategy I districts were low. Labor cost data are not available for this analysis 
because none of the nine TA Liaisons in the sample participated in recruiting Strategy I districts or 
LAs. 

Strategy II districts are more numerous (approximately 20). EMSTAC usually contacted and 
then visited them, which resulted in higher recruitment costs. Please see the EMSTAC 2001 
Evaluation Report and Adjustments^ for a more detailed analysis of costs associated with recruitment 
of Strategy I and II school districts and LAs. 

Strategy III districts (32 of which have completed the on-line training) were recruited largely 
by EMSTAC staff who were attending professional meetings and through strategic partners and 
outreach to such organizations as state departments of education. Some recruitment-related costs are 
common across Strategies II and III, such as EMSTAC staff attendance and presentations at 
professional meetings, contacts provided by our partner organizations (such as CASE), and contacts 
through our website. 

Exhibit III-24 shows the change in labor costs across time for recruitment activities for 
Strategies II and III. Labor expenses for Strategy II decreased across three years but increased 
slightly in Year 5. Recruitment for Strategy II was initiated in Year 2 and decreased across Year 3 
and Year 4. Recruitment activities shifted in emphasis from Strategy II to Strategy III in Year 4. The 
increase in Year 5 is due to the recruitment of three additional Strategy II districts in Year 5. These 
districts were recruited through strategic partners that worked with previous Strategy II districts. 
Labor expenses for Strategy III decreased from Year 4 to Year 5 as EMSTAC entered the final year 
of the contract and ended recruitment efforts. 



’ Mesmer, E., Ritter, S., Paulsen, C., Carl, B., Dailey, D., Shami, M., Hamilton, J., Mclnemey, M., & Gerver, M. 
(2001). The Elementary and Middle Schools Technical Assistance Center: Evaluation report and adjustments from 
Year 4. 
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EXHIBIT 111-24 



Changes in Distribution of Labor Costs for Recruitment Across Time 

for Strategies II and III 




S2 - Yr 2 S2 - Yr 3 S2 • Yf 4 S2 - Yr 5 S3 - Yr 4 S3 - Yr 5 



Training 

As shown in Exhibit III-20, 18.5 percent of the combined total expenses incurred by 
EMSTAC to date can be attributed to training-related activities. The training of EMSTAC LAs 
occurred in different ways across each of the three TA strategies. Strategy I LAs were trained in 
Washington, D.C. in September 1998. As seen in Exhibit III-25, one training was conducted in Year 
2 for all Strategy I LAs. Expenses consisted largely of travel-related costs, consultant fees for 
participating in the training, and EMSTAC staff preparation time. 

Strategy II districts were trained on-site by EMSTAC staff; costs included travel, materials, 
logistics, and labor. Multiple Strategy II trainings took place across Years 2 through 5. Additionally, 
costs were incurred for Strategy II districts that were trained and did not continue working with 
EMSTAC. Exhibit III-25 shows a consistent decline in labor expenses from Years 2 through 5 for 
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training Strategy II LAs. This is due to the shift in emphasis from Strategy II to Strategy III in Year 
4. The majority of LAs who completed training in Years 4 and 5 were recruited for Strategy III. 

Strategy III districts were trained on-line, with no face-to-face visits as in Strategies I and II. 
However, staff labor was necessary in order to encourage the LAs to complete the training through 
online and telephone communication. As more LAs complete the online training, the overall cost of 
Strategy III training decreases. EMSTAC labor costs for training Strategy III LAs decreased slightly 
across the two years of Strategy III trainings. The decrease is smaller than that of Strategy II because 
of the nature of the labor associated with each TA strategy and the variance in timing of LA 
trainings. 

EXHIBIT 111-25: 

Changes in Distribution of Labor Costs for Training Across 

Time Per Strategy 



20 . 0 % 



18.0% 



16.0% 



14.0% 



12 . 0 % 



10 . 0 % 



8 . 0 % 



6 . 0 % 



4.0% 



2 . 0 % 



0 . 0 % 




SI - Yr 2 



S2 - Yr 2 



S2 - Yr 3 



S2 - Yr 4 



S2 - Yr 5 



S3- Yr4 



S3- Yr5 
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Technical Assistance (TA) Support 

As shown in Exhibit III-20, 63 percent of the combined total expenses incurred by EMSTAC 
to date across the areas of recruitment, training, and TA support can be attributed to TA-related 
activities. Costs associated with providing TA to LAs, schools, and districts involve many areas, 
including EMSTAC staff time spent consulting with school personnel, a share of web development 
costs, and time spent developing written and web-based products. The method of providing TA to 
school districts varies across strategies. With Strategies I and II, TA support provided by EMSTAC 
has in most cases been conducted on a personal basis, involving site visits by EMSTAC staff and 
frequent personal communication (by telephone and e-mail) between EMSTAC staff and the LAs in 
school districts. In Strategy III sites, by contrast, TA support is designed to take place using distance 
technology, rather than on face-to-face meetings. A more detailed analysis can be found in the 
EMSTAC 2001 Evaluation Report and Adjustments"^. 

Exhibit III-26 shows the changes in distribution of labor costs for TA Support for each of the 
TA Strategies. Labor costs associated with Strategy I TA support consistently decreased over time. 
Strategy II and Strategy III labor costs associated with TA Support followed a common pattern in 
their first year that differed from that of Strategy I. For Strategies II and III, there was an increase in 
labor expenses from the first to second year of TA Support. Strategy II showed a decrease in labor 
expenses for TA Support from Year 3 to Year 4 and a slight decrease from Year 4 to Year 5. Strategy 
III was only in place for two years; therefore it is not possible to determine if the Strategy II pattern 
would repeat itself in Strategy III. 

A possible explanation for the first year increase for Strategies II and III could be based on 
the time of year when training was conducted. All Strategy I LAs were trained in September 1998. 

Mesmer, E., Ritter, S., Paulsen, C., Carl, B., Dailey, D., Shami, M., Hamilton, J., Mclnemey, M., & Gerver, M. 
(2001). The Elementary and Middle Schools Technical Assistance Center: Evaluation report and adjustments from 
Year 4. 
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Strategy II and III LAs, on the other hand, were trained separately at varying times of the year. 
Strategy II and III sites continued to come on board and be trained throughout the remainder of the 
project period. This may have contributed to the increase in labor associated with TA Support in the 
initial years of Strategies II and III. 



EXHIBIT 111-26: 

Changes in Distribution of Labor Costs for TA Support Across Time 

Per Strategy 




Sl-Yr2 S1-Yr3 Sl-Yr4 Sl-Yr5 S2-Yr2 S2-Yr3 S2-Yr4 S2-Yr5 S3-Yr4 S3-Yr5 



In summary, TA Support is the functional area that consistently incurs the majority of 
expenses across the three TA strategies. As seen in Exhibit III-20 TA Support accounts for 63 
percent of total costs and 71.3 percent of labor expenses. Recruitment and Training each account for 
approximately 18 percent of total costs. 
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While the majority of labor expenses are used for TA Support in all three TA strategies, the 
pattern is not consistent across TA strategy for Recruitment and Training. More labor was spent on 
Training (21.5 percent) and Recruitment (16.4 percent) for Strategy II than Strategy III (Training = 
13.7 percent, Recruitment =15.7 percent). A greater percentage of labor was spent on Training than 
on Recruitment for Strategy II. In contrast, a greater percentage of labor was spent on Recruitment 
than on Training for Strategy III. 

Generally, labor expense decreased for each function over time, across TA strategies. This is 
partially due to the fact that EMSTAC staged the introduction of each TA strategy; Strategy I was 
introduced in Year 2 (of the project). Strategy II began in Year 3, and Strategy III began in the latter 
half of Year 4. This decrease can also be attributed to capacity building at the local level, thus 
requiring less time and involvement of the TA Liaison. 
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CHAPTER IV: 



DISCUSSION OF RESULTS AND LESSONS LEARNED 

In this final chapter, the results presented previously will be revisited and discussed in light 
of findings over the past four years. This review will follow the outline of Chapter 3, focusing on 
recruitment, supports, strategy comparisons, impact, cost, and strategic partnerships. This discussion 
will be followed by a list of specific concluding statements drawn from the evaluation findings. This 
chapter concludes with a more broad based reflection on the lessons learned from these findings and 
the experience of EMSTAC TA delivery over the past five years. 

Recruitment 

The process of delivering TA is complex and requires several steps, including development 
of a professional relationship between the district and the TA entity. Across all three TA strategies a 
significant part of the initial relationship building is the successful recruitment of a district to utilize 
EMSTAC support services. This year’s evaluation found professional referral continuing as the most 
successful strategy for recruiting districts across the three strategies. This reflects the notion that 
successful recruitment in part hinges upon factors such as trust and credibility. Although the 
percentage of Strategy III districts that have been recruited via “professional referral” is somewhat 
smaller than for the other strategies, the districts in this strategy have largely been recruited through 
EMSTAC ’s alliance with numerous strategic partners. This strategy, too, suggests the importance of 
trust and credibility in recruiting a district. Finally, EMSTAC has been moderately successful in 
recruiting school districts through professional development conferences which again suggests the 
importance of credibility, as well as the perceived usefulness of the resources that a center has to 
offer. 
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A significant and fairly definitive finding across the strategies is that “cold contact” 
recruitment strategies were not successful. EMSTAC has recruited very few districts when the 
district had no prior knowledge of EMSTAC and when the initial contact was impersonal (i.e., phone 
call, letter, website). While the website was not specifically targeted as a mechanism to recruit school 
districts, information was provided on the website regarding how a district could get involved with 
EMSTAC. Most school districts are used to seeking out assistance as opposed to being sought after, 
and the influence of legitimacy associated with professional referral plays an important role in this 
process. As a result, some districts may have been unclear about EMSTAC ’s intentions. 

The vast majority of districts that received EMSTAC support have become involved due to 
personal/professional contacts, through strategic partnerships or via professional conferences. These 
districts learned about EMSTAC through a mutual contact or acquired first hand knowledge of 
EMSTAC through a professional presentation or face to face meeting with an EMSTAC staff 
member. Underlying all three of these recruitment methods is the fact that an interpersonal 
connection was made between EMSTAC and the school district. 

The success of professional/personal contacts suggests the importance of trust. School 
districts may be wary of working with an outside support system; however, they may be less averse 
to such support if someone known to the district brokers and supports a relationship with the outside 
support system. School districts often have many types of “outside” support available to them. This 
support can reside through local agencies, state or regional levels, or from a national center such as 
EMSTAC. Frequently, school districts prefer local supports due to their history with and 
understanding of the school system. This factor might place a national TA center at a recruitment 
disadvantage. 

EMSTAC’s participation in professional development conferences and collaboration with 
strategic partners were important in establishing its credibility. Presentations at a professional 
conference provided a platform for highlighting EMSTAC’s expertise. The opportunities for face-to- 
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face interaction may have instilled a level of confidence and credibility that is not easily replicated by 
other recruitment means. EMSTAC's affiliation with strategic partners such as CASE and CCSSO 
also strengthened credibility due to the respect of those groups in their respective fields. EMSTAC's 
work with these groups has been critical in building state-level ties that have often resulted in 
recruitment of local school districts. 

Even when credibility and trust are present, it is possible a district would not be successfully 
recruited if the district did not perceive the services provided by EMSTAC would be beneficial. 
Among those districts collaborating with EMSTAC, the most commonly cited reason for 
participating with the project was its focus on providing access to research-based information. In 
addition, a number of districts reported that they generally felt EMSTAC could serve as an important 
support in helping them move forward with needed change. 

It is important to understand that one of EMSTAC’s goals was to build district capacity to 
solve problems that require research-based solutions. The first “requirement” for accessing 
EMSTAC resources is to complete Linking Agent training, which some districts may have perceived 
as a barrier to services. In such instances, districts may have decided to go elsewhere for assistance, 
particularly if they were seeking immediate solutions to their needs. It is more likely, however, that 
the large number of responsibilities expected of Linking Agents prevented them from engaging in or 
completing on-line training. Related to this is lack of time, the most frequently cited reason for 
delaying the completion of training. 

We have highlighted several interpersonal factors that play a critical role in establishing a 
relationship between a TA support system and its client school district. The development of such a 
relationship begins with recruitment. EMSTAC engaged in two primary methods of recruitment that 
have shown promise in developing strong collaborative relationships. One effective way of recruiting 
districts is through forming strategic partnerships with such entities as professional organizations 
(e.g., CASE) or regional resource centers (e.g., MPRRC). However, reliance on strategic partners to 
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recruit districts can be an expensive method. Thus, the cost of this method may have to be weighed 
against its benefits in fostering initial relationships and successful recruitment. A second effective yet 
less expensive strategy for recruiting school districts is through the use of professional/personal 
contacts. This strategy also promotes the development of interpersonal relationships between 
EMSTAC and school districts, but costs little. The disadvantage of this strategy is that it cannot be 
used as a systematic way to scale up to a larger number of school districts, as it is limited by the 
number of EMSTAC contacts. 

Technical Assistance Supports 

Over the past five years EMSTAC increasingly developed its capacity to provide TA support 
through various means. The support provided by EMSTAC to Strategy I districts continues to be 
highly personal (face to face, telephone) and individualized. Strategy II also included support 
mechanisms that are highly personal (face to face training), but also emphasized use of email and the 
EMSTAC listserv. Finally, with Strategy III, EMSTAC provided support that emphasized use of 
web-based technologies such as the EMSTAC training and TA center website, chat sessions, and 
electronic bulletin boards. 

Linking Agent Training 

While the use of web-based technologies is building the capacity to serve large numbers of 
districts, it is also important to examine the impact of these “virtual TA services” on the process of 
change. Comparable data collected from the Linking Agent training across each of the three TA 
strategies largely focused on the impressions or satisfaction levels of the trainee. While positive 
impressions of the training does not necessarily translate into the implementation of research - based 
practices, this type of data is important because negative impressions of the training might preclude 
Linking Agents from using what they learn from training. 
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Strategy I Training required the cohort of Strategy I Linking Agents convened in Washington 
to receive a two-day face-to-face training program focused on the Havelock Change Process Model. 
The cohort of Strategy I Linking Agents agreed that the experience enhanced their knowledge and 
skills regarding the change process model. In this event, Linking Agents felt that an interactive 
format would have been more beneficial to their learning as opposed to the lecture type format; thus, 
the training curriculum was modified to meet these needs. 

Training for Strategy II Linking Agents was also face-to-face; however. Linking Agents were 
convened in small groups on a regional basis, with approximately six persons in each group. 
Videotapes and interactive exercises were incorporated into Strategy II Training as a way for 
attendees to apply their learning. The training sessions enabled participants to interact and share 
experiences with colleagues. 

Overall, a majority of respondents indicated that the strengths of this training experience 
were the opportunities to participate in group activities, the training and support materials provided, 
and the knowledge of the presenters. Attendees indicated that the relationships established during the 
two-day regional training created a support network they could utilize as they implemented new 
initiatives in their districts. 

During the last year and a half. Linking Agents completed Strategy III training through an 
online, interactive curriculum. Although the content of the on-line training was similar to that used in 
Strategy I and Strategy II training, this forum used video clips, electronic forms, and discussion 
boards to engage the learner in the curriculum, as well as hard-copy support materials and 
“EMSTAC Stack of Support Cards” that contained step-by-step instructions related to accessing 
specific components of the on-line training. 

Trainees completed an electronic training evaluation form immediately following their 
completion of the training. Among the individuals completing the training this year, the vast majority 
said the on-line training gave them the essential knowledge, skills, and confidence to assume the role 
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of Linking Agent. In other instances, Linking Agents reported that the curriculum validated what 
they already knew about change at the local level. Most respondents expressed satisfaction with the 
content of the training material and especially liked the flexibility of time they had in completing the 
modules. 

Recommendations from the Linking Agents regarding improvements to the training 
curriculum primarily pertained to problems Linking Agents experienced in accessing the audio and 
video clips due to their outdated computer equipment. Additionally, we received comments 
regarding the need for more easily downloadable documents. 

Overall, on-line training appears to be an equally effective training format as face-to-face 
training, though there are three caveats to consider. First, motivating prospective Linking Agents to 
get on board with electronic training was more challenging than training face-to-face. At this point 
in time individuals appear to be more likely to access the training system through more traditional 
training formats. Second, once individuals accessed the training program they were not as likely to 
complete training under the Strategy III format. Among other considerations, this has implications 
for the cost effectiveness of this format if less than a critical mass of individuals complete training. 

To address these motivational issues the training format and content must be flexible. Finally, the 
trust and interpersonal connection established between the TA provider and the LA during face-to- 
face training can be especially critical. This connection enhanced the strength of the relationship 
between the LA and the TA Liaison, and thereby provided the TA Liaison with more opportunities to 
support the LA. In some instances, this missing personal connection during Strategy III training 
appears to have made it more difficult to establish positive interpersonal relationships between TA 
providers and LAs, though not in all cases. 
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TA Electronic Support 

EMSTAC electronic support was provided through the EMSTAC website in the form of 
research-based written products and materials, real-time chat events, a bulletin board, the EMSTAC 
Listserv, and the EMSTAC mailbox. During the past year, we added to these supports computer 
based videoconferencing technologies. Further insight into the potential role of web-based support 
mechanisms can be gleaned from data regarding EMSTAC’s website, list-serv, chat-events, and 
bulletin boards. Each Linking Agent who completed training had access to these electronic supports. 
Conceptually, these supports provided Linking Agents with an opportunity to engage in dialogue 
with their peers and to interact with EMSTAC staff and consultants on issues of importance. Strategy 
III Linking Agents were more likely to use these web-based services than Strategy I and II Linking 
Agents. These differences could be a function of a number of factors. Strategy III districts were 
more likely to be attracted to EMSTAC services because of these electronic supports. Further, 
Strategy I and II Linking Agents have become more conditioned to immediate discussion and 
feedback than can be provided through bulletin boards and listservs. 

Strategy Comparisons: The Linking Agent Role 

Linking Agents appear to be engaging in similar types of support activities regardless of the 
EMSTAC strategy through which they participate. Despite the fact that Strategy I districts receive 
significant funding to support their change initiatives, and despite differences between Strategy III 
and Strategies I and II, the nature and amount of work conducted by Linking Agents across those 
districts do not appear to differ significantly. Findings from this year’s data collection show that 
strategies are comparable with respect to their level of LA implementation. Further, we are 
encouraged by what appears to be real and significant improvement in districts using either Strategy I 
or Strategy II over time. These districts continue to make progress in supporting implementation of 
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research-based practices. Though Strategy III districts are generally not as far along, they are also 
making progress in moving research to practice. 

The factors influencing this general pattern of progress appear to be a combination of internal 
and external factors. During the initial years, district level support is important in facilitating Linking 
Agents’ abilities to engage in change agent activities. This is not surprising, particularly during the 
first year, as endorsement by district level leaders such as the director of special education is likely to 
be necessary. Linking Agents will be able to accomplish little if those who have a stake in seeing that 
local needs are addressed do not support them. Further, district leaders often have the power to 
modify job responsibilities so that newly trained linking agents have more time in their schedule to 
engage in these activities. 

The second important factor is the district’s history with past initiatives. Results suggest that 
when the district or school has some experience implementing other research-based initiatives within 
the past five years Linking Agents will be engaged in more “Linking” activities. This is a critical 
finding in that it suggests prior experience within the district in addressing special education issues 
may create a level of readiness or preparedness to deal with new issues. It is also possible that the 
Linking Agents in these districts may have prior experience engaging in change related activities. 
This experience may have served them well in assuming their new responsibilities as LAs. 

Our findings also show that when a problem and need for change was identified at the school 
level and became an impetus for improvement, school staff gain a greater sense of awareness and 
ownership with regard to the initiative. This “grass roots” effort toward improvement takes on an 
inherent legitimacy that is more difficult for external mandates to achieve. 

As the initiative moves into its second year of implementation, support from district leaders 
continue to be important, but support from building level administrators becomes especially critical. 
Despite this pattern, findings this year show a decrease in the degree to which variation in local 
administrator support explains differences in implementation. This could be a function of the lack of 
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variation among districts with regard to administrator support and implementation, both of which are 
high in most districts at this stage. It may also suggest that once the initiative builds momentum and 
capacity, support from district and school administrators is less important. 

What emerges as fundamentally important at this point is collaboration between the LA and 
school staff, including teachers and administrators. The initiatives that stay the course and make a 
difference are those where the LA is able to use EMSTAC research -based supports to build school 
and classroom capacity through teacher knowledge and skills, and those that foster teacher 
motivation to adopt the goals of the initiative as their own. This is a continuing process that usually 
involves extensive collaboration. 

Interestingly, it also appears that districts initially believing EMSTAC would be a beneficial 
resource were most likely to make progress in their change efforts. This finding may reflect the 
relationship between open systems that value input from outside sources and an orientation toward 
continuous improvement. 

Finally, technical assistance plays a key role in moving districts toward implementation of 
research-based programs. District visits by TA Liaisons and phone calls to support LAs, particularly 
during Year 1, appear to help districts achieve greater levels of success. 

Regarding the Stage of Change, EMSTAC evaluation findings over the years have identified 
four variables significantly related to district progress. First, the level of internal collaboration 
between the Linking Agent and other school/district staff was positively related to stage of change. 
The fact that this variable was found to be significant during the first year of working with EMSTAC 
may highlight the importance of collaboration in the planning stage of program/intervention 
development. It is during this stage that input is provided in problem identification and analysis, and 
in the initial attempt at developing/selecting effective interventions. 

The remaining three variables significantly related to the Stage of Change are all considered 
EMSTAC support variables. The first of these. Reason for Becoming Involved with EMSTAC, 
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relates to the recruitment process and serves to confirm the findings discussed earlier in that section. 
It appears that having some prior personal/professional connection to EMSTAC was not only 
important in recmiting a district but was also positively related to the amount of progress that a 
district made during the first year. It is possible that the tmst and credibility important to recmiting 
the district may have served as a springboard to developing strong relationships between Linking 
Agents and TA Liaisons during the training process and throughout the first year of working with 
EMSTAC. In fact, several Linking Agents reported that the close ties with TA Liaisons were 
important in “motivating” and “moving” the process. 

The importance of the strong relational ties between TA Liaisons and Linking Agents during 
the first year of work is further substantiated by the finding that the number of site visits by TA 
Liaisons were positively related to Stage of Change during the first year. This finding suggests that a 
greater number of site visits were associated with greater levels of change during Year 1 . The 
additional attention that TA Liaisons provided to school districts when they made site visits may 
have resulted in a number of important outcomes such as clarifying EMSTAC support, establishing a 
presence in the district, serving as a motivating force to make sure the initiative did not lose 
momentum, and continuing to build tmst and an interpersonal bond. These factors in turn may have 
been important in contributing to the change process. 

Finally, the last variable suggests that the type of TA support provided to a district (not just 
the frequency of support) during the first year was related to the district’s Stage of Change. In 
particular, administrative/logistical support appears to have the strongest relationship to the stage of 
change that a district is in this year. TA Liaisons who provided this type of support often answered 
questions or provided information about financial support, the Linking Agent role, resources 
available from EMSTAC, and information about contacting experts. These logistical type questions 
may have provided concrete information to Linking Agents that allowed them to take specific steps 
toward supporting the initiative. 
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During the second year of work with EMSTAC, internal collaboration and involvement with 
past initiatives remained positively related to Linking Agent activities and Stage of Change, 
respectively. However, during the second year, two additional factors emerged as being related to 
both Linking Agent activities and the Stage of Change; Building Level Support and Locus of 
Change. 

The importance of support from building level leaders in the implementation of new 
interventions/programs was established in last year’s evaluation. The fact that this variable appears to 
be increasingly important in the second year of implementation indicates the significance of building 
level leadership in sustaining the use of research-based practices. Over the course of one or two years 
district/school priorities can change as new needs emerge. Under these circumstances on-going 
commitment and involvement of building-level leaders to previous initiatives can prevent the 
initiative from becoming a fad that is quickly dropped. 

One of the more interesting findings is that Locus of Change was found to be an important 
factor in relation to Linking Agent Implementation and Stage of Change during Year 2. In particular, 
this finding indicates that the change process may be positively impacted when the motivation to 
address an identified need comes from within a particular school building as opposed to external 
forces (district and state). This finding may be more expected during the first year of implementation 
considering that feelings of ownership and empowerment are likely to be high at that time. The fact 
that this factor continued to be important in the second year speaks to the influence of internal 
motivation on sustainability. 

EMSTAC Impact on Linking Agents 

One critical goal that our current evaluation was designed to address is the impact EMSTAC 
had on the districts/Linking Agents with whom we work. Second, this evaluation was intended to 
trace how Linking Agents are able to support those who actually implement the newly adopted 
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research-based practices. These evaluation goals are important for at least two reasons. First, our 
original logic model regarding a national TA center was predicated on the relationships described in 
the above goals. The support that EMSTAC provides to local school districts is dependent on the 
relationship between EMSTAC and Linking Agents, and Linking Agents and local school personnel 
and, therefore, it is important to determine the impact of these relationships at each level. Second, it 
is important to document the resources being provided by a national TA center in order determine 
their effectiveness in influencing educational practices at the local level. The findings reported in the 
impact section of the results chapter, in particular, provide some illumination into each TA strategy’s 
impact at the local level. 

In general, the findings highlight both similarities and differences across the three strategies. 
Prior to discussing these similarities and differences, a quick review of the most salient of these 
findings is presented. 

Similarities: 

• Positive interpersonal relations between TA Liaisons and Linking Agents, supported by 
frequent and meaningful contact, was a fundamental quality in building district capacity to 
support teachers involved in moving research to practice. 

• EMSTAC support appears to gradually diminish each year; however, districts continued to 
make progress in implementing the change initiative. 

• Districts tend to report that EMSTAC’s greatest resource is access to research-based 
information. 

• The most common reason TA Liaisons and Linking Agents communicate was to provide 
EMSTAC with updates regarding district progress on a particular initiative. 

Differences: 

• Strategy I Linking Agents are more likely to report that EMSTAC provides important 
emotional encouragement and process support. 

• EMSTAC’s primary role in working with Strategy I districts has been to assist them in 
figuring out a way to address a particular need. The primary role in working with Strategy II 
sites was to assist them in sustaining or enhancing an initiative that was already in place. 
Strategy III districts represent of mixture of these two patterns. 




IV-12 



147 



American Institutes for Research 



• Strategy III Linking Agents were more likely to take advantage of EMSTAC’s electronic 
supports than Linking Agents in Strategy I and II districts. 

• While positive interpersonal relations supported by frequent contact has proven to be critical 
to maximizing effective TA delivery across all three strategies, by design Strategy III districts 
involve less contact. Some of these districts have reported the need for more structured 
personal contact, as experienced by Strategy I and II districts. 

Overall these findings show that EMSTAC’s impact across the three strategies was similar and 

involves common forms of TA support. One possible reason for the differences is the varying 

expectations that school districts had of EMSTAC regarding types of support. During recruitment 

and training, EMSTAC’s focus has been to impress upon Strategy II (and III) districts that our 

support will assist in facilitating access to research-based information. With Strategy I districts, 

additional emphasis was placed on the importance of the TA Liaison in actually assisting Linking 

Agents in their change efforts. In fact, as reported earlier. Linking Agents perceived EMSTAC’s 

support to be more important to their efforts when TA Liaisons made more frequent site visits. Thus, 

the perception of EMSTAC’s effectiveness may be associated with some level of personalized 

presence of support. A second possibility may have been the training format utilized in Strategy I and 

II. Training for these two strategies were both face to face, yet the Strategy I training may have been 

perceived as a bigger or more important event because Linking Agents (and in some cases 

administrators) were flown to Washington D.C. to meet with EMSTAC and OSEP officials. This 

training may have had an aura about it that served to establish EMSTAC as a legitimate partner with 

each school district. As a result, these school districts may have become more invested in their 

relationship with EMSTAC. A third possibility relates to the significant funding provided to each 

Strategy I district. EMSTAC has served as an important source of financial resources for Strategy I 

districts to implement a change initiative. Although no “strings” were attached to this funding, it is 

likely that the money facilitated greater EMSTAC input into decisions that were made regarding 

implementation and support of the research-based initiatives. A final possible reason may be that 

several Strategy II districts had already identified an intervention and were often in the early stages 
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of implementation when approached by EMSTAC. EMSTAC’s later point of entry in working with 
these districts may have created a different type of support relationship when compared to 
EMSTAC’s relationship with Strategy I districts. 

Linking Agent Impact at the Local Level 

In addition to elucidating the relationship between EMSTAC and Linking Agents, results of 
our analyses provide insight into the role Linking Agents play in facilitating change within their 
districts. The findings demonstrate that EMSTAC effectively trained local Linking Agents to 
identify and support implementation of research-based practices, positively impact systemic change 
efforts at the local level, scale-up research based practices to other schools within the district, and 
solve new problems using research-based solutions. Across each strategy districts are working 
directly with teachers in moving research to practice as a result of LA activity and support. Linking 
Agents in Strategy I districts appear to have more hands-on involvement in providing support, 
coaching teachers, and bringing people and resources together to put research into practice. On the 
other extreme. Strategy III districts in general are, as expected, not as far along in this process. Still, 
we can identify exemplary districts in each strategy where Linking Agents are actively involved in 
bringing about school and classroom level change as a result of the capacity building that occurred 
through EMSTAC support. Where this is occurring, evidence is available to show improvement in 
child outcomes. 

Cost Considerations 

Across the strategies. Strategy I is the most expensive by design, since they received support 
for Linking Agent salary and expenses. Therefore, it is noteworthy that districts in Strategy II and 
Strategy III (which did not receive extra support) were mostly performing just as well as districts in 



rV-14 



149 



American Institutes for Research 



Strategy I. This suggests that financial support alone does not predict successful implementation of 
the change process and the use of research-based practices. 

While Strategy I is more expensive by design, TA costs in general appear highly sensitive to 
the characteristics of the districts that are served and the degree to which they used TA services, 
regardless of strategy. “High intensity” districts or those that requested support on a frequent basis 
were most costly, independent of TA strategy. Our analysis showed that TA Support continued to 
incur the majority of expenses across the three TA strategies. TA Support accounted for 63% of total 
costs. While the majority of labor expenses were used for TA Support in all three TA strategies, the 
pattern was not consistent across TA strategy for Recruitment and Training. More labor was spent on 
training and recruitment for Strategy I and II than Strategy III. Generally, labor expense decreased 
for each function over time, across TA strategies. TA Support begins with more intensive 
involvement of the TA Liaison. As the Linking Agent becomes familiar with the process and their 
role, they typically required less TA support. 

The use of long-distance TA mechanisms, such as web-based technologies, significantly 
reduced the cost of providing TA when large numbers of districts are being provided with support. 
This issue must be counterbalanced with the greater challenge of motivating prospective Linking 
Agents to access and complete electronic training. If a critical mass of districts does not complete 
training and use EMSTAC services this format may not be cost effective. 

Strategic Partnerships 

As a national technical assistance center EMSTAC always recognized the need to move 
beyond an individualized district-by-district approach to recmitment, training, and TA support. 
Developing TA partnerships to reach the thousands of local school districts across the country 
suggests the need for additional recmitment, training, and follow-up TA support strategies. EMSTAC 
has been interested in matching its training and product resources to state and regional-level technical 
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assistance priorities and professional development goals. By partnering with SEAs and other similar 
organizations, EMSTAC can access a larger number of local school districts within a state than by 
individual LEA recruitment efforts alone. EMSTAC maximized access to potential LEA partners on 
a state, regional and local level by strategically partnering with other organizations with established 
LEA networks. 

With these issues in mind, EMSTAC evolved beyond its initial recruitment and training 
approach and worked to establish strategic partnerships both at the SEA level and with organizations 
working at regional, state, and individual LEA levels to provide TA resources. Altogether, 

EMSTAC learned how effective strategic partnerships are developed at varying levels of the system, 
regional-state-local, and how these relationships can enhance TA delivery. 

Evaluation Statements 

The results from four years of evaluations can be developed into evaluation statements that 
build across various findings. These statements are outlined and briefly discussed below. 

• Less Costly Forms of Technical Assistance Appear Effective. Less expensive forms of 
technical assistance and school change appear to be just as strong as more costly forms of 
technical assistance and school change. 

One of the most interesting and potentially important findings from our evaluation is that 
school districts across the strategies engaged in school change activities at relatively equal levels, 
though Strategy III districts entered the project at a later time and are therefore not as far along in the 
change cycle. As scores on the Linking Agent Implementation Scale reveal, little difference exists 
among districts in the three strategies. 

• Technical Assistance Costs Are Largely Driven By Technical Assistance Users. The 
cost of delivering technical assistance is largely dependent on the recipient of the 
technical assistance and less on the technical assistance strategy itself 

The primary cost difference across the three strategies is the funding of the Linking Agent 
salary in Strategy I districts and provision of funding to these districts to bring in outside consultants 
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and pay for other aspects of professional development events. As designed, Strategy I was the most 
expensive technical assistance approach. When controlling for these expenses our analysis shows that 
the cost of TA support within a particular strategy is just as variable as the cost of TA support 
between strategies. The critical cost difference becomes the degree to which local LAs used 
EMSTAC TA services, and highly intense TA users existed in all three strategies. Generally, when 
the cost of paying the Linking Agent stipend was controlled for, the expense of each strategy was 
roughly equal. 

• Scaling Up Is an Interpersonal Process. Scaling up through a national TA system is 
largely dependent on a national center’s ability to develop interpersonal networks with 
other groups who work closely with school districts. 

EMSTAC established positive working relationships with a number of State and local 
technical assistance service providers. Further strategic partnerships were created with parent groups, 
professional associations, and others who support local schools. Through this work, EMSTAC began 
to identify the multiple layers of established TA arrangements and build a network of coordinated 
service delivery. 

• Effective Scaling Up of Technical Assistance Will Utilize Technology. Technical 
assistance that is provided primarily via “distance” mechanisms is likely to play an 
important role in a national center’s capacity to scale up. 

EMSTAC demonstrated the potential for scaling up technical assistance by using long 
distance training and support mechanisms. For example, although EMSTAC was only able to 
feasibly support seven school districts using the first technical assistance strategy, we were able to 
provide support services to a far larger number of districts using technical assistance Strategy III, 
which is “distance-based.” This accomplishment was significant because it established a workable 
model for scaling up technical assistance support to large numbers of school districts. 

• A National Technical Assistance Approach Can Effectively “Teach People to Fish ”. 
Despite decreasing requests for assistance from EMSTAC, participating school districts 
have successfully continued their efforts to improve outcomes for students with 
disabilities, thus demonstrating their capacity to meet their own future needs. 
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During the initial stages of our relationship with school districts, EMSTAC found it was 
necessary to have contact with school districts between two and three times per month. This was true 
for both Strategy I and Strategy II districts. Contact with Strategy III districts was somewhat less 
necessary. Instead, many of these districts directly accessed the website to obtain information as 
opposed to contacting EMSTAC TA Liaisons. For both Strategy I and II districts the number of 
contacts diminished over time as local district capacity, confidence, and momentum was established. 

• EMSTAC has increased the importance that professionals working in local 
education agencies place on research-based practices. By the end of the first year of 
working with EMSTAC, districts reported increased interest in and respect for research- 
based practices. 

EMSTAC utilized a number of innovative technologies to ensure that information regarding 
effective and research-based instructional practices were accessed by professionals within local 
educational agencies. In spring of 1999, EMSTAC began making information available to Linking 
Agents via our website. This began with the establishment of a private Listserv and most recently 
included the addition of live chat events and web-based video conferencing. 

• EMSTAC has demonstrated that information regarding research-based practices is 
permeating the local level. 

Drawing from a variety of data sources, such as interviews, surveys, and observations across 
the years EMSTAC found its services enabled schools and districts to engage in activities resulting in 
implementation of research-based practices. Quantitative ratings are supported by a number of 
interview statements and observations demonstrating that information about research-based practices 
is highly prevalent in these districts. Awareness of this information was critical to moving research 
to practice. 

• EMSTAC has documented that positive change (i.e., the implementation of research- 
based practices) is actually occurring in the districts that are served. 

Moving from awareness of research-based practices to actually implementing these practices 
in schools and classrooms is challenging. It involves a continuing process of building capacity and 
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motivation among teachers and other service providers. Our evaluation data indicate EMSTAC 
districts are making progress in building this capacity and motivation. In all three strategies 
particular districts stand out as exemplary in actually moving research into practice, and the general 
pattern for districts across the strategies is positive. Almost all of the Strategy I and Strategy II 
districts made steady progress in addressing their needs and in building their capacity to meet new 
needs. The progress made from year to year was statistically significant (when considering the mean 
of each strategy as a whole), which indicates the effectiveness of both strategies in promoting the 
change process. As expected. Strategy III districts as a whole tend not to be as far along as those in 
Strategies I and II, though they are making similar progress. 

Interpretation of data analyzed relative to stage of change indicates that districts made 
statistically significant increases from year to year, indicating progress in using Havelock’s model for 
change. 



Final Reflections on Lessons Learned 

The goal of providing a practical, user-fidendly, and effective source of technical assistance 
to local school districts on a national scale is a significant undertaking. Over the past five years 
EMSTAC learned how to overcome many of the barriers and challenges to implementing a national 
technical assistance approach. While not insurmountable hurdles, the challenges reviewed below 
represent important issues that have been considered and addressed as part of the process of meeting 
the needs of local schools and their students with disabilities. The first issue focuses on the 
fundamental ingredients involved in moving research to practice in districts and schools - motivation 
and capacity. These two important conditions are difficult to foster in places where they do not 
already exist, but they are essential to accomplishing the goals of a national TA center. The second 
issue addresses the complexity of working with increasingly large numbers of districts while 
simultaneously providing services responsive to the unique needs of each district. The third issue 
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concerns the struggle that many district and school-based educators face in wearing multiple hats of 
responsibility with numerous time constraints. We conclude by reflecting on issues related to 
identifying the most effective recipients of our Linking Agent training and the most effective partners 
to facilitate the change process. 

Motivation and Capacity 

In order for schools and teachers to effectively implement research-based practices, and make 
progress in change, they must have the motivation and capacity to do so. The same conditions are 
necessary for Linking Agents. As evidenced in our experience and evaluation findings, these are 
fundamental qualities cutting across all aspects of TA. 

We can think of motivation as the will to accomplish something. There are strong links 
between environmental contexts and motivational conditions. A school cannot “grab hold” of a 
person’s thoughts and feelings and manipulate them as if they were molding a piece of clay, but it 
can try to influence motivational processes through environmental levers. Motivation cannot be 
imposed, as people cannot be forced to genuinely care about something or feel a particular emotion. 
Yet motivation can be facilitated or constrained. It is possible to alter the probability that a person 
will adopt or learn a particular practice. Indeed, it is unusual when a pattern of behavior persists in 
the face of compelling contextual incentives, pressures, and supports designed to constrain or 
facilitate change in that pattern. Motivational processes serve as prime targets for TA centers and 
districts seeking to foster commitment to change. 

Based on our evaluation findings we can see that motivation to move research to practice 
among Linking Agents and school-based professionals was achieved when the following factors 
were present: 

> Awareness among school professionals of the change initiative and research-based 
practices associated with this initiative. 
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>• Perceived legitimacy of the initiative and research-based practices among school 
professionals. 

>• Perception among school professionals that they have the necessary knowledge and 
skills to successfully implement the initiative and research-based practices, and that 
their school and district environment will support them in this effort. 

> Perception among school professionals of a sense of ownership and empowerment 
with regard to the goals of the change initiative and associated research based 
practices. 

Even when teachers and other school-based professionals come to adopt all initiatives as their 
own, and develop sufficient awareness and appreciation of research-based practices designed to help 
them implement the initiative, a number of issues of capacity still need to be addressed or these 
efforts will fall short of their goals. 

Capacity is usually defined as the ability to do something, and school capacity ultimately can 
be measured in terms of a school's capacity to add value to desirable student outcomes (however 
student outcomes are defined). We can build on these ideas to envision capacity as consisting of four 
components that can be applied to both schools and classrooms: human capital, physical capital, 
community networking and social capital, and organizational culture. Though each of these variables 
is important, they are likely not of equal weight, and their relative importance will vary with the 
context and mix of factors present at each school site. 

Human Capital 

Human capital refers to the intellectual ability, knowledge and skills of individuals working 
in districts and schools. Some variables, such as teacher content knowledge, have immediate and 
direct effects on the quality of instruction. Students represent another important element of human 
capital. Student knowledge and characteristics directly impact how resources and activities are 
arranged to enhance learning. 



IV -21 



American Institutes for Research 



Physical Capital 



Physical capital relates to a wide range of resources potentially available to schools. 

Financial capital provides the source of fiscal assets upon which schools can draw to support their 
ongoing activities. These assets are available for purchasing other forms of capital. Physical 
resources also include buildings, technology, instructional materials and supplies, instructional time, 
class size, and school size, which may have indirect and long-term effects on student learning. 

Community Networking, Social Capital, and Trust 

Districts and schools are comprised of a rich diversity of individuals and groups. In order to 
achieve a high level of productive cohesion, individuals within the district and schools must be 
linked in a strong network of trust. Further, individuals must be oriented to the common good of the 
group as a whole. Trust of and support for the common good are promoted by a district or school 
community characterized by positive interpersonal relations and collaboration. 

These factors of motivation and capacity emerge from the EMSTAC findings and provide 
useful targets for future TA services. 

Breadth vs. Depth 

Each of the three EMSTAC training strategies present their own sets of barriers and 
challenges. A significant advantage of utilizing the face-to-face training approach developed for 
Strategies I and II was the ability to establish a personalized working relationship with Linking 
Agents in each of the participating local districts. A personalized approach was effective in guiding 
Linking Agents beyond the initial training stages of the EMSTAC partnership. The inherent 
disadvantage, however, of such a labor-intensive approach was in the limited number of LEAs it was 
possible to train and provide with on-going technical assistance. Under Strategy I, EMSTAC trained 
and provided follow-up support and financial resources to seven LEAs. Under Strategy II, an 
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additional 15 LEAs received face-to-face training and ongoing technical assistance. In order for 
EMSTAC to move significantly beyond this number of LEA partners, the web-based training and 
support resources utilized over the past two years in Strategy III became necessary. 

As EMSTAC began implementing Strategy III, the challenge of breadth became less of a 
factor. The potential for far greater numbers of LEAs to partner with EMSTAC under this strategy 
was significant. For example, since September 2001, there have been over 3,500 individual visitors 
to the EMSTAC public website. Linking Agents logged onto the Website and accessed training 
modules and other support resources according to their own schedule and for any length of time that 
was convenient. Despite this progress, as this strategy evolves it becomes more challenging to 
provide quality resources, support materials, and on-going communication with local educators 
responsive to their specific needs and circumstances. 

Selecting Effective Linking Agents 

The process of determining the most appropriate local educator to take on the responsibilities 
of being an EMSTAC Linking Agent has been an ongoing learning process. Many of the district and 
local school leaders who appear to be most appropriate for this role have turned out to be somewhat 
ineffective. For example, many local directors of special education and school principals, upon 
receiving EMSTAC information, indicate a high personal interest in being trained and working as 
Linking Agents. In practice, most of these individuals have proven unsuitable for operating 
effectively as Linking Agents due to their leadership responsibilities. The best role for administrators 
seems to be one of leadership, where they can designate a staff member to be the Linking Agent and 
provide ongoing support for the Linking Agent role within the district. Furthermore, designating a 
team of school district personnel to work with the Linking Agent is often desirable. This team can 
help sustain momentum and more effectively solve complex TA problems. We recommend to 
administrators that staff development specialists, curriculum specialists, special education 
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coordinators, and resource teachers effectively fulfill the Linking Agent role. A challenge, 
particularly in smaller districts, in identifying a person to serve as a Linking Agent is the shortage of 
appropriate personnel. Some of the local education agencies in our “prospective district” recmitment 
category were smaller districts with one or two administrators who have multiple responsibilities and 
an insufficient number of personnel available to accept the Linking Agent role. 

Helping administrators find ways to effectively manage their time in order to incorporate 
school improvement efforts into their array of responsibilities was an important challenge as 
EMSTAC strived to meet the needs of local school districts. This challenge seemed especially 
pertinent as EMSTAC increased the number of Strategy III districts. Administrative leadership and 
guidance was especially critical in districts where EMSTAC support was provided at a distance. 

Selecting Appropriate Targets and Partners 

Throughout the history of EMSTAC, we primarily selected and partnered with education 
officials at the local school district level, as required in the original RFP. This effort met with 
moderate to good success, with many school district officials embracing our technical assistance 
approach by stating that it holds much promise compared to the usual top-down models previously 
implemented. For example, many local officials stated that the usual top-down, one-shot-workshop 
model produced, at best, temporary or no change at the local level. What seemed most appealing to 
local districts about EMSTAC ’s insider-outsider model was our use of a Linking Agent or district- 
based support team that responded to the needs of local educators, and worked with district staff until 
the desired research-based practices were implemented and institutionalized. Local district personnel 
indicated that the most positive value of the insider-outsider. Linking Agent model was that the local 
Linking Agent and support team members developed a plan that addressed the unique needs of the 
school district, the context and history of effort, and provided a means for sustaining the identified 
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interventions by continuing to work toward improving school practices over a significant period of 
time. 

While the primary target audience for EMSTAC’s work has been the local school district, we 
also partnered with a wide variety of other agencies. Our work with agencies beyond the local level 
was initiated with the goal of promoting an active and sustained effort for research-based school 
improvement across multiple entry points. These additional agencies have included state 
Departments of Education, Intermediate Education Units (encompassing state technical assistance 
systems and individual technical assistance providers), parent organizations, regional resource 
personnel (including within and across State regional technical assistance providers), and other 
national organizations dedicated to providing in-service training activities and technical assistance 
events to educators and families. EMSTAC partnered with these groups in recognition of their 
important role in delivering technical assistance to schools and with the goal of providing 
complementary resources. 

Several findings have emerged from our work with multiple partners. In general, the most 
successful approach to the partnering process includes both an open-system attitude (in which a large 
number of potential partners are recognized as potential conduits for an expanded technical 
assistance effort) and a flexible system (in which potential partners can be included or bypassed as 
appropriate to the goal of effective, efficient delivery of technical assistance to schools). Our 
experience suggests that while many local. State, and national-level agencies and organizations want 
to partner with EMSTAC to facilitate the delivery of technical assistance at the local level, at least as 
many have struggled with incorporating EMSTAC resources into their sphere of activities and 
influence. Strategy III especially lends itself to partnering with others as part of the scaling-up effort. 
However, this opportunity also presents a challenge due to the limited number of face-to-face 
interactions when TA is provided at a distance. 
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The primary lesson learned from EMSTAC efforts to partner with State officials, State 
technical assistance entities, national organizations, and local school systems is that technical 
assistance is best delivered to schools through a variety of entry points, largely dependent on the 
specific needs, priorities, and work climate of the local and State education agencies. Depending on 
the climate and contexts encountered, successful partnerships were established by EMSTAC at each 
of these levels. The key challenge for EMSTAC was to maintain enough flexibility to identify those 
agencies and organizations that were truly interested in upgrading services and outcomes for students 
with disabilities at the local level and to provide resources that were complementary to their efforts. 
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SCHOOL DISTRICT QUALITATIVE DESCRIPTIONS 

District 1 



District Background 

The Linking Agent (LA) was interviewed via phone call in May 2002. District 1 serves a 
population of approximately 1 1 ,000 students in about 20 public schools, ranging from Kindergarten 
through twelfth grade. Eighteen percent of the student population receives special education services, 
59 percent receive free or reduced price lunch, and 4.8 percent are minorities. 

As a Strategy I school district. District 1 has been working with EMSTAC for the past four 
years. During the first year, 1998-1999, the LA worked with the Technical Assistance (TA) Liaison 
to complete LA training and identify district needs. This led to the selection of programs in literacy, 
specifically reading and writing, for students in the elementary grades of one particular school in the 
district. The LA and school principal collaborated to develop a plan for implementing strategies from 
the Early Literacy Program (ELP) developed at Michigan State University. These included Morning 
Message, a daily reading and writing activity used with Kindergarteners, and POWER, a writing tool 
used in the upper elementary grades. In addition, the school participated in several other initiatives 
including a state Reading Network, Character Education, and Correlates of Effective Schools. The 
District 1 schools have also developed and implemented inclusion programs across the district for 
students with emotional disturbance. This has been facilitated with wraparound principles and the 
implementation of positive behavior supports. District I’s work with outside researchers included an 
initiative during the summer of 1999, when the LA facilitated training with a researcher from 
Michigan State University in efforts to implement the ELP strategies at the targeted school. 
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During the 1999-2000 school year, the LA working with the TA Liaison to implement 
another new initiative, the Strategic Instruction Model (SIM) program. SIMs is a strategy instruction 
program, designed at the University of Kansas, to help students with study skills, reading 
comprehension, and self-monitoring skills. The program targets students in the upper elementary 
grades, and was selected because it was seen as a useful tool for students included in general 
education classroom settings. Teachers who participated were trained in differentiated and 
individualized learning techniques. 

The LA and TA Liaison also began work to implement a program focusing on social skills 
training for all students; especially those included in general education classrooms through inclusion. 
Records at the targeted school indicated that many student were being suspended for aggressive 
behavior that violated the rights of others and school and district policies. The program. Second 
Steps, was selected that year and was implemented in one elementary school by the end of the first 
year. 

Initially, the LA spent much time with teachers and administrators, planning trainings, 
modeling techniques, coaching, and providing general support. During the past two years, however, 
the need for support and coaching has dwindled, and she has been less hands-on in the schools with 
regards to implementing these efforts. During the 2000-2001 school year, the LA narrowed her focus 
to scaling up the SIM and Second Steps programs. Second Steps was funded by a State Improvement 
Grant, designed to assess why students were being suspended in targeted schools and investigate how 
students might better develop and maintain appropriate social skills. District 1 facilitated a training of 
trainers and set up training for teachers and developed technical assistance goals for Second Steps, as 
well as an evaluation plan. Although this effort was not funded through EMSTAC, the two previous 
efforts for District 1 had been. The EMSTAC TA Liaison was involved throughout the process as a 
resource for advice as well as access to relevant research. 
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Stage of Change 

This year, the LA has been spending much of her time working with a team to implement the 
Second Steps social skills program district-wide. Although this effort began during the 2000-2001 
school year, it became a much more significant part of the LA’s responsibilities this year. All 
elementary school teachers in the county have been trained in Second Steps, impacting about 150 
classrooms in 14 elementary schools in the county. Funds and technical assistance supporting this 
effort have come mostly from the state, although the EMSTAC TA Liaison has provided some 
assistance with planning. The LA has been critical to this process, providing training to teachers, 
coaching, conducting classroom observations, and collecting and analyzing data for formative and 
summative evaluations. So far, there has been little follow up support since the summer training. 
However, it is the hope of the LA to continue working to support teachers implementing these 
programs, and to plan much needed follow-up activities. 

Also during the 2001-2002 school year, the LA worked with teachers to expand the use of the 
SIM program, which had begun during the district’s second year of the project. The program is now 
being used in four high schools and four middle schools. It is not, however, used as a school-wide 
program in any of the schools. The LA also facilitated teacher professional development in three new 
components of the SIM programs. A local consultant trained teachers during a summer program in 
the three intervention strategies: SLANT, self-questioning, and the unit organizer. The LA is also 
working closely with secondary supervisors in the district in helping them make the transition to full 
inclusion in middle and high schools. 

It seems that the efforts of the LA have come full circle with regard to following the 
Havelock model. The district was in the “Renew” stage last year, as the LA had moved on from the 
ELP strategies work on the implementation of SIM. This year, the district has implemented and is in 
the “Try” stage with new SIM strategies. The district has also focused more on the scaling up of the 
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Second Steps program, which was also supported through their State Department of Education. This 
effort is currently in the “Extend” stage. The LA has successfully implemented three interventions 
and has successfully scaled up one intervention. Although two interventions were not scaled up to the 
extent originally intended, they are examples of how helpful research-based programs can be to 
schools when they are selected and implemented in a meaningful and targeted fashion. 

Important Factors 

District I’s support stems from many streams, including external technical assistance and 
internal support. External technical assistance comes from the State Improvement Grant, Frostburg 
State University, the University of Kansas, Johns Hopkins University, and EMSTAC. The LA relies 
on these organizations for information and research-based practices, especially in the areas of 
literacy, strategy instruction, social skills programs, and inclusion. She has also relied on their 
training and consulting services. Internal supports have included the former principal at the 
elementary school implementing the ELP strategies, and her supervisor in the District office. This 
support has allowed the LA flexibility with her schedule, provided some financial support, given her 
time to research ideas on the internet, attend conferences, work with EMSTAC, and read professional 
journals. This is in line with her supervisor’s continued commitment to infusing research-based 
practices into the curriculum. However, the LA is concerned that after the financial support from 
EMSTAC is gone, she may not be able to continue performing in her current capacity. Although 
there is support for her current role, the shift in focus of district priorities may demand that she work 
in other capacities. 

Other important factors include the rapport between the LA and TA Liaison, and the LA’s 
willingness to jump in and work with both district and school level personnel. She has a wide 
knowledge base and a strong relationship with many important players in the district. She has been 
dedicated to her role as an LA, and has taken her mission to heart. This has clearly played a role in 
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her success as a facilitator of change and a key factor in the implementation of various research- 
based programs in the district. 

EMSTAC Support 

The LA utilizes EMSTAC’s resources to identify research-based practices related to the goals 
of District 1, based on the district needs assessment. As she indicated in her interview, she has 
participated in a few of the chat events, receives information from her TA Liaison, communicates 
through the listserv, and accesses information presented on both the public and private side of the 
website. She feels that the website could be an even more valuable tool, if it is accessed by those who 
are in a position to readily use that information. 

The LA has expressed that over the four years, her need to access EMSTAC and the support 
of the TA Liaison has decreased, not because EMSTAC is not valuable, but because she feels 
EMSTAC’s services and supports were most valuable during the first two years of her work as an 
LA. Over time, she feels she has gained a better understanding of her role as a change agent and feels 
more confident in carrying out her duties as such. Her training and support from her TA Liaison in 
the first and second year of the project helped establish a foundational approach for her future 
endeavors as a change agent. She feels that students in her district, especially those with disabilities, 
have been touched by these efforts. 
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EXHIBIT A-1: 

District 1 Technical Assistance 
Delivery Model, 2001-2002 
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Indicates a strong relationship 
Indicates a moderate relationship 
Indicates an existing relationship 
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District 2 



District Background 

On April 29, 2002, two EMSTAC staff members conducted a telephone interview with the 
Linking Agent (LA) from District 2. The LA is a school-based resource teacher at a middle school 
and has been working in District 2 for more than 25 years. She has been working with EMSTAC for 
the past four years on the goal of reducing discipline problems and continues to work closely with 
students, parents, teachers and administrators in her capacity as an LA. 

District 2 is a large, urban school district with a student enrollment of 167,074 students. 
Minority groups (predominantly African-American) represent 96.3 percent of the student population. 
Approximately 70 percent of the students in the district receive a free or reduced-price lunch and 
about eight percent of the total student body receive special education services. The middle school 
that the LA works in has a student population of nearly 805 students (grades six through eight), 99 
percent of the student body is African-American and 63 percent of the student body receive free or 
reduced price lunch. Further, about 15 percent of the student body (122 students) receives special 
education services in the school. 

District 2 was the first urban school district to be recruited by EMSTAC in 1998 and has 
worked with EMSTAC as a Strategy I district for the past four years. After meeting with the local 
Director of Special Education and the Special Education Constellation Supervisor, the district 
decided to work with EMSTAC and identified a potential LA and pilot school for the intervention. 
Based on an internal needs assessment conducted prior to beginning to work with EMSTAC, the LA 
chose to focus her efforts as an LA on school-wide discipline. In interviews conducted the first year 
that EMSTAC was working with the LA, most teachers and administrators at the middle school 
agreed that student discipline was the most important need at their school and that no other needs 
should take precedence. 
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After identifying the need, the LA worked with EMSTAC and the Special Education 
Constellation Supervisors to select an appropriate intervention for improving student discipline. The 
intervention selected was Project ACHIEVE. This program is a school-wide intervention that 
addresses students’ social skills, teachers’ behavioral management skills, and the school’s capacity to 
use a problem-solving consultation framework for developing new academic and behavioral 
interventions. EMSTAC worked with the LA to implementing Project ACHIEVE in one middle 
school in the district. The LA began by implementing the STOP and THINK practices in all the 
classrooms. During the second year of working with EMSTAC, she focused on “trying to have each 
class review the steps” and created a library of videos focusing on the various social skill lessons in 
the program. This approach to reviewing the social skills was effective with both teachers and 
students. The LA commented that the “Teachers really enjoy it. Kids even come up to me and ask 
why we haven’t seen one [a video] in awhile.” Based on collected data, the program has proven to 
be success and the LA reported a marked reduction in the number of office referral and suspensions. 

Over the years, there have been a number of interesting and important findings that have 
resulted from EMSTAC’s work with District 2. One of the initial findings was the importance the LA 
attributed to the support of the special education supervisor. The close relationship between the LA 
and the supervisor helped to ensure that the program remained a priority among the administrators at 
the district level. Further, we found strong building-level support from the principal and teachers for 
both addressing the need of school-wide discipline and, also, for the initiatives themselves. For 
example, the LA said that “the principal of this school believes that it is a priority, and he supports 
it.” Most of the teachers agree — student discipline is the most important issue facing the student 
body — and, as a result, believe that these programs are suitable for their needs and goals. 

Another significant finding relates to the impact of the increasing independence and 
confidence of the LA throughout the implementation of the project. Initially, the LA relied heavily on 
the consultant and on the EMSTAC TA Liaison for support in implementing the program on school- 
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wide discipline; however, as the LA has become more familiar with her activities and responsibilities 
as an LA, she has been able to continue to implement the program effectively as support is gradually 
decreased. The increasing independence of the LA has also led to the expansion of the program to 
include new research-based strategies. Finally, the school district has a number of ongoing initiatives 
for improving outcomes for children in special education and are open to change. The impact of these 
initiatives has often supported, complemented, and supplemented the work that the LA is doing with 
EMSTAC. 

Stage of Change 

District 2 is still addressing its original need of improving school-wide discipline, but the LA 
has changed the strategies used to address this need. During a conference that the LA attended with 
two TA Liaisons from EMSTAC, the LA learned about the “Promoting Positive Behavior” program. 
This school-wide discipline program focuses on creating a positive atmosphere within the school and 
rewards students for their positive behaviors. This program recognizes the positive behaviors that 
students display rather than the negative behaviors. The LA believed that this program would be 
effective in their school and began implementing the program this past fall. Every week, the LA 
distributes nomination forms to all the teachers in the building. Each teacher can then nominate 
students for the “student of the week” award. Students are nominated for displaying numerous 
positive behaviors such as exhibiting improved social skills, being polite, respectful and 
hardworking, displaying an improved attitude and effort, or finding a wallet and going from class to 
class to find its owner. Both the nominees and the winners are rewarded through public recognition 
over the PA system and also given a small prize. 

The LA believes that “this is a good program to recognize the positive rather than the 
negative” and that “it [expanding the program] would be nice to see.” The LA believes that it is 
necessary to expand the program to the elementary “feeder” schools in order to really make a large 
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impact in the social skill development of the students in the district. Unfortunately, due to the 
numerous changes at the district level, the LA does not anticipate the district will provide financial 
assistance to support the expansion of this program next year. Therefore, although the LA is 
attempting to move this initiative to the “Extend” stage of the Havelock model, barriers have 
impeded the progress. Rather, District 2 continues to work on the “Try” stage of the Havelock model 
with the LA making adaptations and changes to the programs to ensure that the initiatives are 
meeting the needs of the teachers and students in the school to improve school-wide behavior. 

Important Factors 

It appears that one of the most significant factors contributing to the success of the program 
has been the building-level support. The majority of the teachers and the students have been very 
enthusiastic about the program this year. The students often ask if they have been nominated and 
enjoy hearing their names read over the PA system. Although this is a new program for encouraging 
positive student behavior, the LA has often heard teachers and students using the Project ACHIEVE 
language they have learned over the past two years. The majority of the teachers have “bought-into” 
the program and participate in the weekly nominations. According to the LA, these teachers believe 
that this program positively impacts student behavior in the classroom. Some of the teachers are very 
diligent and committed to the program and the teachers that chose not to participate are a small 
minority in the school. The principal also continues to support this initiative because of its common 
goal with Project ACHIEVE — to reduce the number of discipline problems in the school and 
support positive school-wide behavior. For the purposes of evaluating the programs, the LA 
continues to keep track of the students’ discipline files and has planned to examine and compare the 
referral rates at the end of this school year to the referral rates last year and also to the referral rates 
prior to working with EMSTAC. 
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In contrast to the building-level support, the district has provided less support for the 
initiative this year than in previous years. The LA commented that the district is very committed to 
raising standards and they are working for continuous improvement and to include parents more. Yet, 
due to financial problems in the district, there have been no district-level resources provided for the 
implementation of this program. The LA has raised money for the small rewards students win 
through the student store and, when necessary, has used her own money to support the program. As a 
result, the program was not expanded this year to additional schools in the district. The continual 
state of change at the district level over the past year has impacted the ability of the district to support 
this program. The LA did note that although there have been three executive directors this year alone, 
the district administration building is being closed, the employees are being moved to various other 
buildings, and the school board is changing — there does appear to be a strong commitment through 
all the changes to special education and discipline issues. 

EMSTAC Support 

The LA in District 2 has now been working with EMSTAC for the past four years and, with 
each passing year, she has felt more confident in fulfilling her role independently. Initially, the LA 
relied heavily on her EMSTAC TA Liaison and they were in continual communication. EMSTAC 
provided the necessary support to bring the Project ACHIEVE consultant to District 2 to administer 
training and to support the LA in going to a school where Project ACHIEVE was being implemented 
in order to observe its effectiveness. During these first years of working with EMSTAC, the LA said 
that it was very important to her that EMSTAC was always there to provide support and was only a 
phone call away. Last year, the LA commented, “In the beginning, there was a lot of support. I know 
more now and it’s a lot easier.” In our interview with the LA this year, she reported even more 
confidence in supporting her role as an LA herself and rarely communicated with her TA Liaison 
throughout the year. The majority of the contact the LA had with EMSTAC during this past year was 
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for a presentation she and two other TA Liaisons prepared for a conference. The LA presented with 
two EMSTAC TA Liaisons about the success of Project ACHIEVE and the work that she was doing 
with EMSTAC. She shared in our interview how grateful she was for the opportunity to present at a 
conference. “It was a beautiful experience for me.” 

In addition to the personal support the LA has received through her communication with 
EMSTAC TA Liaisons, she has utilized the various electronic resources of EMSTAC. She has found 
the listserv to be very helpful and although she might not reply to the messages, she always prints 
them out, reads them, and disseminates them to persons that she believes would benefit from and 
appreciate the information. She also uses the information on the public and private EMSTAC 
website. The “Social Skills and Discipline” section has been particularly useful in identifying 
resources appropriate for addressing their need. She has been unable to participate in the on-line chat 
events due to technical difficulties but has downloaded all the transcripts from the chats and has 
learned a lot from the discussions. 

District 2 has a history of being committed to improving the educational opportunities within 
the schools for all children, including children with special needs. Over the past five years they have 
implemented several initiatives that focus on reducing school-wide behavior problems and violence 
in the schools, improving standards and achievement, and inclusive practices for students with 
special needs. This is a district that embraces change and looks for programs that can help the 
students succeed but this is also a district that has been experiencing a lot of change in the 
administration and school board and has been facing financial problems. These barriers to 
implementing new initiatives have slowed the work of the LA in expanding the effective program 
that she has implemented through her work with EMSTAC. 

Despite these barriers, the LA is hoping to continue her work as an LA next year. She plans 
to keep implementing the “Promoting Positive Behavior” program in her own school and is still 
hopeful of expanding the program. Eventually, she would like to see the program expanded to the 
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other middle schools in the district and the elementary, “feeder” schools but does not anticipate this 
will happen within the next year due to the numerous changes in the district staff and with the school 
board. 




A-13 



182 



American Institutes for Research 



EXHIBIT A-2: 

District 2 Technical Assistance 
Delivery Model, 2001-2002 
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District 3 and District 4 



District Background 

District 3 and District 4 are both now in their fourth year of working with EMSTAC. They 
began as Strategy I districts in the 1998-99 school year. Both districts are continuing to focus their 
efforts on improving reading instruction and outcomes in the early grades and at the middle school 
level. This year, District 3 has initiated some new projects around this goal and both districts are 
continuing to build on what they have previously put in place over the past few years. 

While both districts are situated near each other within the state, the demographics of each 
district are distinct. District 3 is a larger district, considered to be in a mid-size urban area according 
to the National Center for Education Statistics. Its schools serve 18,502 students, 12.3 percent of 
whom receive special education services. 15.5 percent of this district’s students are from culturally 
and linguistically diverse backgrounds. Approximately 20.2 percent qualify for free or reduced price 
lunch. 

The District 4, in contrast, is a much smaller community. The 1,216 students attend four 
schools (Primary (K-3), 4-6, Middle, High). Of these students, 1 1.4 percent are identified as special 
education and 10.4 percent qualify for free or reduced price lunch. About 4.7 percent are from 
culturally and linguistically diverse backgrounds. 

Stage of Change 

Both districts are continuing to work on implementing phonemic awareness initiatives to help 
achieve their goals of improving reading instruction and prevent reading failure by using research- 
based strategies. In District 3, the four sites focusing on phonemic awareness are now in what the 
Linking Agent (LA) terms the “Maintenance” phases or the Havelock “Extend” stage - personnel in 
the districts have already been trained and so now just check in with the LA on a regular basis. This 
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year, the LA did train one more teacher who was new and he continues to provide support to her. 
District 4 is also in the “Extend” stage. The LA remains in contact with the staff to see how things 
are going and also holds “catching up” sessions about 2 or 3 times a semester. 

District 3 is also working on several additional initiatives both on a school-wide and district- 
wide basis. The middle school is continuing to implement on a school-wide basis Peer Assisted 
Learning Strategies (PALS). There is also one elementary school focusing on collaborative strategic 
reading. The LA visits these schools once a month to observe and work with them on these 
interventions. On a district level, the LA is in his third year of working with a cadre of teachers 
trained in reading comprehension strategies. This cadre helps to train other teachers throughout the 
schools. A new component has been introduced where each teacher involved selects a colleague to 
become trained and be involved which expands the group each year. In addition, the LA has been 
responding to calls to assist with improving beginning reading skills and trainings on explicit phonics 
instruction and examining the criteria for basal readers. Due to the extreme budget cuts in the district 
this year, the LA has been assisting schools in figuring out the best way to utilize primary grade 
teachers in the most effective manner. 

While the LA has been busy in both districts, his role has primarily been a maintenance role 
in District 4 while in District 3 his role has also extended to help with new initiatives the schools 
wish to implement. The LA, in commenting on his work with the various schools in the districts, 
noted, “It is gratifying that this has taken on a life of its own.” In District 3, since it is still a relatively 
small district, word of mouth (especially through principals) has been the primary method of 
dissemination of the resources available to the district through EMSTAC and the LA. In addition, a 
brochure was created this year to help explain and advertise the cadre of teachers and a teacher who 
was previously at the middle school doing PALS has since been transferred to an elementary school 
where she continues to use it. Since District 4 is such a small district, there really has not been as 
much room for expansion to occur. 
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As Strategy I sites, both districts have been receiving funding to implement the interventions 
the LA is working on. With EMSTAC in its final year of its contract, the LA has began working with 
both districts to find ways to insure the initiatives continue beyond the end of their association with 
EMSTAC. Ensuring the continuation of these initiatives is further complicated due to state budget 
cuts as well as some changes in personnel in both districts. 

In District 3, the LA has been fortunate to work extensively this year with the district’s 
Curriculum Coordinator who is very excited and committed to the work that has been going on. She 
will continue the work currently going one once the contract with EMSTAC is over. In addition, the 
LA has arranged a meeting with the outgoing and incoming principals of the middle school in order 
to discuss what has been accomplished through EMSTAC and to stress the importance of sustaining 
the PALS program already in place. 

Similarly, in District 4 the LA is working to ensure that the work done thus far through 
EMSTAC is sustained. He has selected one teacher who has been very involved in the 
implementation of the phonemic awareness initiative and is working with her to give her additional 
training and information. This way, as new staff are hired in the district, she can train them and 
answer questions as they arise. 

Important Factors 

One important factor that has facilitated both districts’ progress is the LA’s professional and 
personal qualities. He is very committed to his work and enjoys what he does which shows through 
in interactions with him. He has been able to develop solid relationships with both school and district 
level personnel, which has helped in the interventions’ success. 

Support from local district administrators has also been a key factor in the districts’ success. 
In District 3, the LA receives a lot of support from the Curriculum Coordinator. In addition, the 
principals and teachers in the schools have been very supportive and positive towards the initiatives 
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and programs he has helped to implement. The LA has a similar degree of support in District 4. The 
principal in the primary school is a great instructional leader who understands the link between 
instruction and assessment. The principal has made it a priority to ensure that all teachers, aides, and 
even parents are trained in phonemic awareness. 

Another important factor for the success of these districts, as noted by the LA, is the financial 
support that each district receives. He noted that regardless of where the funds come fi"om, they are 
very needed to successfully implement these types of initiatives. He noted that in District 3 the 
Department of Special Education has been giving him $10,000 a year for materials, substitutes, 
stipends, xeroxing, and other small expenses. He commented that without those funds he does not 
think EMSTAC would have been nearly as successful. 

EMSTAC Support 

Due to the relationship that has been established between the LA and EMSTAC over the past 
four years and the experience the LA has gotten over the years, this year the LA did not feel he 
needed a lot of assistance fi-om EMSTAC during the 2001-2002 school year. His most common 
interaction with EMSTAC this past year has been in the form of EMSTAC requesting information 
fi"om him. He still contacts EMSTAC whenever he hits a roadblock and finds his TA Liaison to be 
very helpful and supportive. 

This year the LA got involved with the videoconferencing aspect of EMSTAC. In fact, he 
found that due to EMSTAC’s “technical support,” the videoconferencing, was most beneficial in 
facilitating technical assistance in his districts this year. He says that he has learned a lot about the 
technical aspects and about how to use it. He even helped to facilitate one of the videoconference 
events that was held across multiple districts. Unfortunately, he was not able to participate in any of 
the other videoconferencing events because the time in which they were held takes place in the 
middle of the school day when he is normally in the classrooms. 
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The LA is very supportive of EMSTAC and their mission and this attitude has carried over 
into his work with District 3 and District 4and to the degree of success both districts have had with 
their initiatives. The LA is working diligently to ensure that the success these districts have been 
having in the past four years continues into the years to come. 
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EXHIBIT A-3: 

District 3 and District 4 Technical Assistance 
Delivery Model, 2001-2002 
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District 5 



District Background 

On May 1, 2002, EMSTAC staff conducted an interview with the Linking Agent (LA) from 
District 5, for an end-of-the-year evaluation. 

This District enrolls about 36,885 students, of whom approximately 14 percent represent 
minority groups, 12 percent are students with disabilities, and 24 percent are eligible for free or 
reduced price lunch. District 5 is one of seven districts being supported by EMSTAC through 
Strategy I. This school district was one of the first seven sites to receive training and support from 
EMSTAC. In recent years, the district has experienced a large growth in its general student 
population (enrollment in schools has increased 35 percent since 1990), increasing the challenges 
faced by the school staff in providing high quality services for all students. 

Several major findings came out of last year’s evaluation of EMSTAC’s work in this District. 
First, the role of the LA has grown substantially: She has become an integral member of the feeder- 
wide special education team (consisting of principals from three elementary schools, one middle 
school, and one high school). Simultaneously, with this expansion in her role, the LA has become 
less dependent on EMSTAC, as noted by both the LA and her Technical Assistance (TA) Liaison. A 
second finding was the importance of building-level administration to the successful implementation 
of an innovation. Changes in school principals have directly impacted the time and resources 
allocated to the work done around inclusion — recently, with the new principal feeling involved to 
the project, there has been increased commitment to ensuring that staff had the appropriate time to 
meet and plan regarding the initiative. Finally, there was a “scaling up” of interventions in the district 
— the LA began to plan for ways to bring the inclusion model to more schools in the feeder pattern, 
and the middle school in the feeder pattern identified improving writing skills as their primary need. 
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As indicated in the most recent needs assessment conducted by District 5 in 1999, district 
personnel were concerned with deficits in writing skills of students with disabilities at the middle 
school level. This concern was fueled by an increasing demand to meet local and state assessment 
requirements, and to improve high stakes testing scores. The needs assessment also indicated that the 
district had concerns in regards to developing behavior intervention plans and conducting functional 
assessments. Their previous needs assessment advocated for the need for greater and more 
meaningful inclusion of students with moderate to severe disabilities. 

When the District began working with EMSTAC, inclusion was their top priority. 

Researchers and consultants were then identified to work with the district. Aside from the LA there 
was one special education teacher involved. As of last year, a cadre of teachers at one elementary 
school has been trained and receives support in collaboration around the area of inclusion. The 
collaborative teaching and planning model for inclusion has involved supporting teams of teachers 
who work with individual students with disabilities. The teams set aside time to meet regularly and 
discuss and plan instruction for each child. The LA has weekly meetings with the collaboration teams 
as well as with the principal. While there have been attempts to involve other schools with the 
collaboration model, most of the LA’s efforts focused on the original elementary school largely 
because teacher turnover created a need for new training in the collaborative model. 

With respect to the new need identified in fall 1999 — improving writing skills of students 
with disabilities — the district asked consultants from the state’s Writing Project to conduct training 
during which several strategies for teaching reading and writing were presented. During these 
meetings, journaling was the most appealing strategy, and teachers began to implement it in their 
classrooms. The LA’s efforts were focused on providing support during training, and also working 
closely with instructional assistants to make sure that they were involved meaningfully in the efforts 
to incorporate journaling into classroom activities. 
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Stage of Change 



As mentioned in the previous section, when District 5 began working with EMSTAC, it 
identified greater and more meaningful inclusion of students with moderate to severe disabilities as 
the most significant priority. EMSTAC has been working primarily with one elementary school 
around this need; however, the LA is working to expand the innovation to other schools in the 
feeder-wide pattern. In 1999, a middle school in the feeder-wide system showed an interest in being 
involved with EMSTAC. To this effect, they identified deficits in writing skills of students with 
disabilities at the middle school level as their need. The LA and EMSTAC began to address this need 
during the 2000-200 1 school year. 

Around the issue of inclusion, the collaborative model at the elementary school is now 
school- wide. Despite recent turnover with teachers, what started with two students and one teacher 
has expanded to incorporate special education teachers, general education teachers, music, art and 
physical education teachers, teacher assistants, and other related services personnel. Most staff at the 
elementary school has been trained and is receiving support in collaboration. The teams vary 
depending on the needs of each student. However, each team is interacting on a regular basis and 
finds it easier to plan for and relate to their students. The assistants meet every Tuesday, and they 
meet with the teachers on another day to plan and ensure that the child can be involved in class 
activities. Their meetings include discussions of the child’s Individualized Education Program (lEP), 
the child’s educational goals, and the child’s curriculum. 

The LA sees her role as primarily one of providing organizational support. As a result, her 
involvement is far less than it was in previous years: At the beginning of the school year, she was 
visiting the school on a daily basis to meet with teachers and teacher assistants. After the first six 
weeks of the school year, this dropped to two or three times a week to primarily attend group 
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meetings. Additionally, the LA developed and provided in-service training to special education 
assistants around the issues of inclusion and collaborative planning and teaching. 

The LA feels that the collaboration model is working well especially with the general 
education teachers. They see the value in having the special education students fully involved. In 
terms of evaluating the impact of the intervention, the important thing for the LA is to see whether 
the goals and objectives on the child’s lEP are being met. That measure is important to assess 
whether or not the collaboration and teaming are working. The buy-in from teachers, and the students 
benefiting, has increased so dramatically that it appears that the collaboration is successful. 

When District 5 began working with EMSTAC on improving inclusion, they were at Stage 2 
of Havelock’s change cycle model: “Relate,” which involved establishing contact and relationships 
within the system. Now, after four years working on collaboration, they are in the final stage, 
“Renew.” The collaborative model has been expanded and renewed as more teachers are realizing the 
importance and value of collaboration and of including special education students. 

With respect to the newer need in the District — writing — the LA had asked the state’s 
Writing Project to assist the district in identifying strategies. Several teachers at the middle school 
articulated that writing was a top priority in response to a needs assessment that the LA and the 
school principal had administered. Consultants from the Writing Project came into the school and 
conducted several training sessions where they introduced various strategies for teaching reading and 
writing. During these meetings, journaling was identified as the most appealing strategy, and teachers 
began to implement it in their classroom. The LA provided support at the training sessions and also 
worked closely with instructional assistants to make sure they were involved meaningfully in the 
efforts to incorporate journaling into classroom activities. 

According to the LA, the entire school is journaling, and the program has been incredibly 
well received. The teachers are conducting daily journal-writing activities throughout the middle 
school. To paraphrase the LA, “students love to share what they have written.” In fact, even in the 
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“Learning for Life” classrooms (for students with severe disabilities) journaling has been 
successfully implemented — with some accommodations. Additionally, an elementary school in the 
feeder pattern has chosen to implement journaling in their classrooms. District 5 is still evaluating 
whether they are meeting their goal of improving student writing in the middle school. Because 
MSPAP testing is not available to them anymore, they are using less formal mechanisms of 
collecting and evaluating writing samples to see developments in student writing skills. Like the 
LA’s role in the elementary school intervention, the beginning of the school year was the busiest. She 
was meeting with the Writing Project staff and school administration every other week to strategize 
about how to best implement the initiative. Since then, her involvement has developed into become 
more impromptu. 

In relation to Havelock’s change cycle, the middle school began at Stage 1, “Care,” where 
they began to think seriously about the needs of their students, and they are currently at Stage 6, 
“Extend.” They are looking for ways to expand the initiative into the school and district (an 
elementary school in the Feeder-wide pattern has demonstrated an interest in implementing 
journaling). 

Important Factors 

Several important factors have ensured the District’s success. First and foremost, the LA has 
been a major driving force with both the writing and inclusion initiatives. Other important factors 
include supportive leadership, district and school buy-in, collaboration between the LA and school 
staff, and EMSTAC support. 

Throughout EMSTAC’s work with the district, the LA’s commitment and knowledge have 
been recognized both within the schools and outside as a key part of their success. The TA Liaison 
believes that “she is the entire reason for the successes the county has made over the past 5 years.” 
She maintains excellent rapport and involvement with the staff and students in her district. 
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Furthermore, the LA is flexible and recognizes when she needs to take a leadership position in a 
situation and when she needs to be less involved. With both the inclusion and the writing initiatives, 
the schools are at a point where they are able to work independently, and the LA has taken more of a 
backseat role. However, to get to that point, they needed the organizational support from the LA — 
which she was willing and able to provide. 

Another important factor was the support received from leadership — both at the school 
principal level and the district level. At the elementary school, a new principal has made a big 
difference in the extent to which the initiative has been successfully implemented and expanded. In 
the initial years of implementation, the elementary school encountered several barriers, primarily 
because there was a pretty oppositional principal. He moved on and was replaced by someone who, 
despite being originally a building-level administrator with no background in special education, has 
become increasingly invested to the area of inclusion, collaboration, and teaming. In fact, the 
principal is a facilitator for the special education feeder pattern meetings, demonstrating his level of 
commitment. At the middle school, the principal works closely with the LA and with the teachers 
and instructional assistants. The LA finds him extra conscious of special education needs, providing 
significant support for the special education team. He believes that what the LA is doing is valuable, 
and is committed to supporting her activities. 

In addition, the LA has been incredibly supported at the district level — the Special 
Education Director and the administrator have shown both interest and commitment to the LA’s 
initiatives. 

Related to the support received from leadership is the importance of school buy-in: The LA 
finds that a key lesson learned from implementing the inclusion initiative is the importance of 
establishing relationships with key players: building administrators, school principals, teachers, and 
teacher assistants. She has found that especially with teachers being so busy, it helps when you have 
something to offer them, whether it is services, support, or resources. In fact, the LA makes it a 
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priority to involve them in decision-making, working from bottom-up. She recognizes the importance 
of school staff taking ownership of sustaining and renewing new initiatives. Taking this one step 
further, if the teachers are supporting an initiative, it creates added value for the administration and 
principals. 

Another important factor was the collaboration between the LA and district staff The LA 
makes every effort to attend teacher planning meetings and training sessions. She works 
collaboratively with both teachers and instructional assistants to make sure that staff development is 
meeting their needs, that their questions and concerns are being addressed, and that they are feeling 
involved in the change process. 

Finally, EMSTAC support has been important to ensuring Distirct 5’s success. The LA 
indicated that there were few resources beyond EMSTAC support allocated to this project, and 
expressed an interest in having EMSTAC continue: “It provides the district with opportunities and 
resources. And, it is flexible in that you can choose what topics are useful and of interest to you.” 

This idea of flexibility ran through much of what the LA felt about EMSTAC: initially, she 
recognizes that she used EMSTAC far more than she did during the 2001-2002 school year. As her 
knowledge and confidence in the areas of inclusion, collaboration, and writing grew, the level of 
contact she needed to maintain contact with the TA Liaison decreased. At the same time, there is 
implicit acknowledgement of the continuous support received from EMSTAC: “if I ever felt like I 
needed help, I know whom to call.” 

EMSTAC Support 

EMSTAC ’s relationship with District 5 has been central to their success in implementing 
new initiatives. As a Strategy I site, the District continues to receive financial support during the 
2001-2002 fiscal year. The total budget included funds for the LA’s salary, fringe benefits, and 
technical assistance expenses such as release time for planning activities. The money has been 
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especially important for ensuring that collaboration for inclusion and training with the State Writing 
Project were able to take place. 

The LA describes her relationship with EMSTAC as being extremely helpful when it comes 
to identifying needs, locating research-based materials, facilitating support and follow-up, and 
motivating change. Despite her contact with the TA Liaison being low (their contact mostly consists 
of touching base or catching up), the value of knowing that the Liaison is available is high. In fact, 
while she sees an immense overlap between her job as an LA and a Special Education Facilitator she 
is unequivocal about wanting to continue serving as an LA for the District. 

The LA reported that she really enjoys using the Listserv. While the teachers she works with 
do not have the time to check it regularly, she sees the value in the guidance and resources it 
provides. Another aspect of EMSTAC she values is the accessibility of EMSTAC staff — both the 
TA Liaison and other project staff. Chat events were helpful, but hard to access due to timing and 
technical difficulties. However, the LA frequently went back to the EMSTAC website to access 
transcripts of the chats. Finally, the LA finds EMSTAC products useful and informative. 

Specifically, she shared the product on selecting an intervention with her group of special education 
facilitators. 

The model for technical assistance in District 5 continued to evolve this year, developing 
stronger and more complex connections. The relationship between the LA and the principal and staff 
at the elementary school has continued to grow but, in addition, she has fostered an equally strong 
connection with the middle school principal and the middle school staff. There has been less reliance 
on outside consultants and trainers this year, along with less contact with EMSTAC. In addition, the 
LA’s involvement with the Feeder-wide team continues to grow — they have become more 
interested in implementing best practices at their own schools — and the LA provides the team with 
resources and connections she has developed through her work with EMSTAC and her schools. 
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EXHIBIT A^: 

District 5 Technical Assistance 
Delivery Model, 2001-2002 
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District 6 



District Background 

District 6 serves approximately 9,750 students. Minority students make up 82.3 percent of the 
overall student population. Approximately 3 1 percent of the students enrolled in District 6 participate 
in the free or reduced price lunch program. District 6 is a Strategy I district that was recruited as a 
result of a professional referral (i.e., professional contact, and agency referrals). Contacts made at this 
level directed EMSTAC staff members to decision-makers within the district that could approve and 
commit to a collaborative relationship. Currently two EMSTAC Linking Agents (LAs) are working 
with schools in this District. 

District 6 has made great strides since the inception of its literacy initiatives and collaboration 
with EMSTAC. Three years ago, district officials organized two four-day summer workshops to 
introduce their new reading initiatives. Under the direction of a Reading Consultant, a decision was 
made to convert the past site-based reading curricula to a common core, balanced literacy 
curriculum. During this workshop, each school worked on strategies and means of implementing the 
new balanced literacy strategies. While all schools were interested in formally integrating a new 
literacy program, the Assistant Superintendent selected three schools to pilot a program based on the 
following factors: history of standardized test performance, current curriculum components, free or 
reduced price lunch population, accessibility to financial resources, level of school staff enthusiasm, 
and willingness to implement the new strategies. 

After extensive research and review of the available programs, the three schools chose to 
implement the Waterford Early Reading Program (WERP). The WERP is a software-based early 
intervention literacy program. The program attempts to improve educational outcomes for students 
by addressing student literacy challenges at the outset of their educational career, and integrating 
components for parents, teachers, and students. It includes materials (audiocassettes, videocassettes, 
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and books) for students to take home, thereby bringing literature materials into homes that may not 
otherwise have books for students. With the WERP, teachers have a continuous picture of each 
student’s performance and instruction level. The program tracks each student’s progress, alerting the 
teacher of the relative strengths and areas in need of improvement. This additional information can 
better inform the lesson plans and the concepts that need to be reinforced. Finally, using the WERP, 
students are exposed to a daily interactive literacy experience. Developed for students as early as 
kindergarten, the WERP provides three levels of reading instruction for students. The program 
adjusts to student successes and setbacks, providing an individualized experience for each student. 
Students have the opportunity to achieve success and to increase their skills at a manageable, 
challenging pace. There are also three levels of programs (Reading Readiness and Emergent Skills, 
Beginning Reading, and Fluent Reading) to ensure an experience that will continue to challenge 
students as their skills become more sophisticated. 

Stage of Change 

Last year District 6 sharpened their focus to concentrate on a developmentally appropriate, 
early literacy intervention for all students as a means of reducing future referrals to special education. 
The Waterford program is intended to serve as the all-encompassing improvement to the district’s 
literacy curriculum, while the institution of their newest component is intended as an intervention for 
early elementary students. The desired result is significantly fewer referrals to special education 
when this cohort reaches the third grade. This year, the LAs are still working in the three original 
schools; however, they have expanded the program to include not only the first cohort of students 
who are now completing the first grade, but also a group of students who are currently completing 
kindergarten. From the beginning, the LAs had planned to move the program from kindergarten to 
first grade in order to follow the same cohort of students. After a year of implementing the WERP, 
kindergarten teachers protested moving the program from kindergarten to first grade and called a 
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meeting with the principal to try to prevent this from happening. Because District 6 had made a 
commitment to EMSTAC to follow the cohort of children, the program was still moved to the first 
grade; however, the Elementary Supervisor was able to secure another grant and used these funds to 
maintain the program in the kindergarten classes. 

According to the Havelock model of change, the data suggest that District 6 is currently in 
the “Extend” stage. During our collaboration the District, EMSTAC has worked very closely with 
them to implement research-based interventions with which the district feels comfortable. District 6 
has progressed through the stages of change in a systematic, logical manner. When our collaboration 
began. District 6 was in the “Relate” stage, attempting to define how to integrate the potential 
contributions of EMSTAC and the LAs with the goals in the district. Over time, the TA Liaison and 
EMSTAC have worked with the district through changes of leadership, defining the issues and 
suggesting possible avenues of change, unveiling a new balanced literacy curriculum, and now re- 
framing the focus to concentrate on early intervention. Last year the district worked to develop a plan 
and began to gather information on the results of the intervention. They have made a commitment to 
their intervention, and are putting their research into practice. 

Based on their success, the LAs would like to expand the program to other schools in the 
district, but there are two challenges to overcome before this can happen. First, there is an issue of 
equity. The Supervisor of Curriculum and Instruction feels that it is unfortunate that there is a 
program that is showing positive outcomes for three schools, but the other schools (and students) are 
denied access to the intervention. She would like to see the other schools have the opportunity to 
have access to the WERP as well. To do this, however, there would need to be a commitment to find 
and secure the resources to finance such a venture. This leads to the second issue, the need for 
additional resources. As previously stated, the kindergarten teachers who had the WERP system in 
their classrooms did not want to give the program and computers to the first grade teachers and 
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wanted to keep the system in their own classrooms. At this point and with EMSTAC ending, there 
are no funds set aside to sustain the intervention past this school year. 

Important Factors 

There were three recurring themes in the data for District 6 that suggested their importance in 
the delivery of effective technical assistance, and the district’s progress: (1) EMSTAC as a means of 
external financial support, (2) EMSTAC’s access and provision of research-based information and 
strategies, and (3) the collaborative relationship between the LAs and their TA Liaison. 

Both LAs cited the impact of EMSTAC’s financial support as critical. During the course of 
the collaboration, EMSTAC has paid for staff development, consultants, substitute teachers, and the 
Waterford Program. The constituents in this District feel secure knowing that EMSTAC is able to 
fund their highest prioritized project. District 6 was able to secure an Excellence in Equity grant in 
order to maintain the program at the kindergarten level. Without EMSTAC’s financial support, they 
may have had to find an alternate intervention, and may have had to use valuable time and resources 
to find grants to fund it. 

EMSTAC’s access to research and research-based practices has greatly benefited the district. 
The TA Liaison has been a source of information on a range of ideas related to their topic. He was 
particularly useful in his critical analysis of the Waterford research articles and helping the LAs 
become informed consumers before purchasing the program. He reviewed articles and provided 
written feedback that prompted more thorough research by the district. This dialogue challenged the 
LAs to consider additional aspects of the research and ultimately motivated them to contact the 
program developers to get additional research. 

The LAs felt that the working relationship that they have been able to develop with the TA 
Liaison has been vital to their success. The LAs reported that although the frequency of contact with 
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their TA Liaison has faded over the last year, they still know that he is available should any questions 
arise or if they simply need to “touch base.” 

Each of these variables will be vital for the sustaining and expanding the program. The 
importance of EMSTAC’s financial support cannot be understated. As the primary external financial 
resource, EMSTAC has afforded District 6 the opportunity to have this sophisticated technology- 
based intervention in their pilot schools. Inherent in this intervention is an array of costs (hardware, 
software, network connections, training, technical support, etc.), most of which was made possible 
with the assistance of EMSTAC. As the district tries to sustain and expand the program, the district is 
faced with the need to find expansion funds and replace the original funds provided by EMSTAC. 
The access to research-based practices and interventions is also important to the district, as it helps to 
make them more informed consumers of information. Should the project continue and District 6 
decides to enter the “Renew” stage of Havelock’s model, they are in a position to make better 
decisions when confronted with the research on new topics. 

EMSTAC Support 

The nature of the relationship between the TA Liaison and District 6 has changed during the 
course of our collaboration but is still viewed as important. The LAs maintain a strong relationship 
with one another and the District, as they serve the district as a team. In the first year of 
collaboration, the TA Liaison had a higher level of involvement with the LAs because their need for 
assistance was at a much higher level. There was frequent contact and there was a more frequent 
exchange of information between the LAs and the TA Liaison. Because the program is now “up and 
running” the LAs do not feel that they require as much support as they did in the past. They stated, 
“any time we needed our TA Liaison, he was available for us. He was great!” 

During this past year, the TA Liaison conducted one site visit and has been in contact with 
the LAs through either e-mail or telephone about once every two months. The most common type of 
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interaction with the TA Liaison has been for the purposes of catching up and completing 
administrative tasks. Although the LAs are members of the EMSTAC listserv, they indicated that 
they do not have time to participate and respond to the information posted. They also have not 
participated in any of the EMSTAC chat events or used any of the on-line resources. They view the 
most helpful aspect of EMSTAC’s support as financial. “EMSTAC gave us the financial boost we 
needed to be able to afford supplies, materials, and training that were critical to the Waterford 
Program.” 
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EXHIBIT A-5: 

District 6 Technical Assistance 
Delivery Model, 2001-2002 
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District 7 



District Background 

District 7 is one of the largest school districts in the country. During the 2001-2002 school 
year, the district had a total enrollment of 735,058 and approximately 80.7 percent of the students are 
comprised of minority groups. Special education services are currently being provided to 
approximately 1 1.3 percent of the student population. 

EMSTAC began its partnership with District 7 in June 2000 with one elementary school. 
Because one of the seven original Strategy I districts had recently stopped working with EMSTAC, 
District 7 joined as a replacement for this district. The Linking Agent (LA), working in the former 
district, recruited District 7 through a personal connection he had with the principal at one of the 
elementary schools. The demographic makeup of the school is somewhat different than that of the 
district. Eighty-four percent of the students are Hispanic and 12 percent are Asian. In addition, 60 
percent of the students are English Language Learners. 

On April 30, 2002, two EMSTAC team members conducted a phone interview with the LA 
from the District 7 for annual data collection. This district is a Strategy I school district and technical 
assistance has been concentrated at one elementary school within the district. Therefore, the 
interview questions were specific to the initiative in place in this particular school. 

Stage of Change 

As stated in last year’s report. District 7 is implementing the Peer-Assisted Learning 
Strategies (PALS) program. PALS is a peer-guided collaborative reading strategy that aims to 
increase reading fluency and comprehension. Student pairs take turns orally reading designated text. 
While one student reads, the other student records the number of words read and points out any 
words read incorrectly. After reading the text, each student is required to summarize what they read 
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and report the main idea of the passage. During the past year the LA has been working in the 
elementary school approximately one week per month to continue support of the PALS program in 
grades 3-5. 

The LA provided further support this year when he conducted an in-service training on PALS 
by modeling the strategy to two new teachers. The LA lives at a distance from the school district. 
When he is on-site he works individually with the teachers, observes them, helps to problem solve, 
conducts “advanced” PALS training, and offers ideas on how to create variations of the program. 
When he is not on-site, he is available to the teachers by e-mail or phone. Most of the grade 

teachers are regularly implementing PALS in their classrooms and enjoy using the program. The LA 
has noticed that a few teachers are resistant to the program and do not appear to use it regularly with 
their students. 

When District 7 first began its partnership with EMSTAC, the school was at Stage 1 of 
Havelock’s change cycle, “Care,” in which they had identified a need and were interested in 
addressing it. By the end of Year 1, they were at Stage 5 “Try,” in which they were implementing an 
intervention and testing its effectiveness in meeting their need. Now, at the end of their second year 
of implementation, they are attempting to “Extend” (Stage 6) the program. This is particularly 
evident in that the LA has had all five of the Regional Superintendents, along with their assistants, 
observe PALS in action. Additionally, the LA worked with EMSTAC to set up a videoconference 
event to model the PALS program for other school districts across the country. 
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Important Factors 

Two factors have been important in assuring success with PALS and EMSTAC technical 
assistance. The first is the LA himself The second was the collaborative relationship between the 
LA, the principal, and teachers in the school. 

Even though EMSTAC is coming to an end, the LA remains optimistic that the PALS 
program will be sustained in this school. While the fact that this LA came from outside the system 
might in other instances be considered a barrier, in this case, it was not. In fact, this fall he is 
relocating to an area closer to the district and will continue to work in the school on another project. 

A second important factor was the high level of collaboration that occurred between the LA 
and all of the staff. In addition to his monthly visits, the LA was readily available through e-mail or 
by phone to answer teachers’ questions and provide support. As previously mentioned, while on-site 
he stopped by and observed each classroom, and worked individually with teachers to address any 
problems or concerns they had. 

Unfortunately, the building principal is retiring this year. Currently, it is unclear whether or 
not the next principal will provide the level of leadership necessary to continue PALS. 

EMSTAC Supports 

Although the LA for District 7 no longer communicates with his TA Liaison on a regular 
basis, he views his role as an LA as, “the best job I’ve ever had.” He attributes his infrequent contact 
with EMSTAC to the fact that he no longer needs the level of support that was provided by his TA 
Liaison in the beginning. Even though their contacts have grown less frequent over time, the LA still 
feels that the TA Liaison is available on an ongoing basis if he needs him. This year the majority of 
his communication with EMSTAC has been to establish videoconferencing capabilities to model the 
PALS program. This required a high level of support to test and conduct the videoconference. He 
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noted that the most helpful aspect of EMSTAC’s support this year was the technical support for 
videoconferencing. 

Other aspects of EMSTAC that have been helpful to the LA include information shared 
through the listserv. He has not used any of the on-line products or participated in any chat events 
with the exception of the chat that he facilitated. He also said that without funds from district-level or 
external sources, he did not believe that EMSTAC would have been as successful. 
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EXHIBIT A-6: 

District 7 Technical Assistance 
Delivery Model, 2001-2002 




Exhibit Key 








w 


Indicates a strong relationship 
Indicates a moderate relationship 
Indicates an existing relationship 




W 

^ 







(f^OPV AfE 



ill fc 



A-41 



211 



American Institutes for Research 



District 8 



District Background 

District 8 lies just outside of a large city and is comprised mostly of suburban and urban 
fringe neighborhoods. Currently, the district has a student enrollment of about 107,000 students. 
Approximately 40 percent of these students are minority students, almost 13 percent of the district 
student population receives special education services, and 29 percent of students participate in the 
free or reduced price lunch program. 

EMSTAC became involved with District 8 as a result of a relationship that developed 
through attendance at a number of conferences related to the topic of disproportionality between an 
EMSTAC staff member, who is an expert in disproportionate representation, and the Linking Agent 
(LA). The LA, who was initially hired by District 8 to oversee the implementation of the district’s 
initiatives, disproportionality, and frequently attended conferences about disproportionate 
representation in order to learn more about research programs that were being implemented in school 
districts across the country. The district’s decision to focus on this issue arose from data collected 
and analyzed by the Office of Civil Rights (OCR) in 1996. According to their findings, the district (a 
Strategy II site) had a large problem with over-representation of minority students in special 
education programs. The district, along with five other counties in the state, then entered into a 
contract with OCR to address this problem. 

After reviewing various research-based strategies, interventions, and techniques for 
addressing disproportionality, the district level leaders decided that the Instructional Support Team 
(1ST) Model was most appropriate for their school because this model included an instructional 
aspect to the intervention. The primary purpose of this model is to decrease the number inappropriate 
referrals of minority students to special education by presenting a standardized approach to referral 
for special education evaluation and training teachers to identify and implement effective, research- 
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based training strategies in the general education classroom for students experiencing learning or 
behavioral problems. The model also supports teacher training in cultural and linguistic diversity and 
fosters the development of foundation skills for collecting, analyzing and evaluating student data. 
Prior to working with EMSTAC, the district had implemented the 1ST model in four pilot schools 
that had volunteered to participate in the initiative. 

The 1ST teams are made up of teachers, parents, psychologists, and other specialists, such as 
guidance counselors, reading specialists, social workers, or speech therapists. They meet regularly to 
receive training and to discuss specific students. This model promotes the success of students by 
meeting their needs (educational, behavioral, and social) in the general education classroom. The 
success of the 1ST model depends on identifying and using the natural support systems built into the 
school system, beginning with the child’s classroom, moving to a grade-level team, and ending with 
a school-wide team. The end goal is to meet the needs of diverse learners in the general education 
classroom by providing teachers with workable solutions and school-wide support. 

The LA believed that EMSTAC would be a great potential resource to her as she worked to 
implement the 1ST model and pre-referral strategies in more schools throughout the district. The LA 
also believed that EMSTAC would be a very helpful in “identifying effective practices and good 
programs” and “networking with other districts.” To support the expansion of the program from four 
schools to 17 schools (15 elementary and 2 middle schools), the LA participated in EMSTAC’s 
strategy II training with 20 1ST members in December 1999. After completing the training, 

EMSTAC worked with the LA to identify research-based materials, to provide support and follow-up 
assistance, and to motivate other school personnel throughout the district to participate in the 
program. At the end of the 2000-2001 school year, the LA collected outcome data on 1ST in order to 
evaluate its impact. Based on the information collected, the 1ST program has a 79 percent success 
rate — 79 percent of the students referred to the 1ST committee are currently being adequately served 
in the general education setting and do not require further testing for special education eligibility. 
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The 1ST process has also resulted in a 66 percent parental participation rate at the initial meeting and 
a 60 percent rate of participation at the three-month progress meeting. 

Stage of Change 

Over the past year, the LA has continued to implement the 1ST program. There has not been 
any expansion of the program to new school sites within the past year. Rather, the LA has been 
focused on continuing to refine the existing 1ST model in the 17 schools and promote its success. The 
LA is engaged in a number of activities to “spread the word” about the program. First, the LA met 
with the school board and has given a presentation about the initiative and has encouraged discussion 
on continuing to expand the program to new schools. The LA shared the results of the data she had 
collected and analyzed with the board to show the progress in the pilot schools. After this meeting, 
the LA indicated that the school board began to revisit how they were addressing the issue of 
disproportionality with an aim to focus on the issue more objectively. Second, the LA is promoting 
the implementation of a more established objective criteria set for selecting the schools to participate 
than in the past. Third, teachers and principals in the school district have been hearing about the 
initiative through the “grapevine” and many have contacted the LA with questions and have asked to 
participate in the program. Fourth, during the yearly meeting for new assistant principals, the LA has 
presented information about the 1ST model and has explained how it is implemented. This meeting 
often results in a number of questions about the program and the interest level for the program rises. 
Fifth, the LA is currently working on the development of an informational tri-fold for parents to 
increase the outreach effectiveness of the program. Finally, the LA has been working to secure 
continued funding for the initiative in the schools that have been implementing the program and also 
funding for expanding the program to new schools. In the first two years of implementation of the 
initiative, the district has received a State Special Education Grant to be used for training and 
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materials. These start-up funds were essential to beginning to implement this initiative. New funding 
sources must now be identified. 

In addition to promoting the expansion of the 1ST model, the LA has been busy continuing to 
train new teachers and other school personnel in the 17 1ST sites. The LA typically facilitates a 
training for new teachers which includes topics such as differentiated instruction and curriculum 
modification. For teachers who have been working with the 1ST model, the LA conducts classroom 
observations, offers suggestions and feedback for improvement, produces a monthly newsletter to 
report on the progress of the initiative, and offers quarterly “tips for teachers.” 

The combination of the efforts by the LA situated this district in both the “Try” and “Extend” 
phases of the Havelock change process. 

Important Factors 

District-level leadership and support appear to be an important factor in the progress of the 
1ST program. In particular, one school board member has been very supportive of the program and 
has brought in additional supporters. Further, the new superintendent has made the evaluation of this 
program a priority and has had regular and on-going contact with the LA about the 1ST model. The 
LA and the district have also worked with their Regional Resources Center, which provides 
informational reports and has developed training materials that the LA has used in the pilot schools. 
This past year, the LA was assigned to work under the Psychology Services Department where she 
has experienced a great deal of support. She regularly meets with the director of the Department to 
discuss the program and reports that she finally feels like she is where she belongs in the district. 

Another factor that has contributed to the implementation of the 1ST model in District 8 is the 
strong internal collaboration among the LA, the 1ST teachers and teams, and the administrators 
within the school district. This positive relationship leads to an LA who communicates regularly with 
all key stakeholders, helps to train teachers, and observes classroom practices at least 2-3 times a 
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year. Because of the positive relationship that the LA has built, the 1ST teachers at each school feel 
free to call on the LA to discuss difficult cases and to seek assistance in identifying pre-referral 
interventions. The LA is then able to use the relationship she has with EMSTAC to provide the 
teachers in her school district with information relevant to their specific needs. 

Further, District 8 has benefited from the financial resources provided by both the district and 
the state over the years. These resources include one full-time position for the 1ST coordinator (the 
LA), one half-time position for an 1ST teacher in each of the schools, to increase psychological 
services, and additional money for professional development. Additionally, a grant was provided to 
the district for the first two years of implementation in the amount of $50,000 for additional supplies, 
materials, and resources and another grant was provided that supported the funding for Excellence 
Psychology personnel for the initiative. 

Finally, the openness of the district at large and the individual school personnel participating 
in the initiative have contributed to the success of the work of the LA. This openness to change has 
allowed the LA to introduce numerous new strategies and materials that are research-based. 

EMSTAC Support 

Over the years, the LA has received support from EMSTAC in identifying research-based 
materials, facilitating support and follow-up, and motivating others to extend the program. The LA 
“likes to have the support and the connection to resources” that EMSTAC provides. The LA 
expressed that she likes to discuss issues from a research perspective and EMSTAC provides this 
type of support. She also reported that EMSTAC “helps me look out further than my small little area. 
1 can bring research-based practices to our teachers.” 

Over the past year, the LA has maintained contact with her TA Liaison about twice per 
month. The communication has primarily been through e-mail or phone; however, the TA Liaison 
did make three or four site visits to work with the LA. The LA expressed that she really appreciates 
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the personal touch that EMSTAC provided. According to the LA, the primary reason for contact with 
EMSTAC is to request information from the TA Liaison and gather more information about 
research-based programs and materials. The materials sent to the LA by her TA Liaison have been 
especially helpful as the LA prepared for her presentation to the school board and continued in her 
outreach efforts. She also shared, “I appreciate the access to contacts that EMSTAC provides in 
addition to the transcripts of the web chats that provide access to experts in the field.” 

The LA made full use of the EMSTAC resources. In addition to her personal contact with 
EMSTAC through her TA Liaison, the LA used the various electronic resources that EMSTAC has 
to offer. For example, she used the on-line training to train several new 1ST teachers and 
psychologists. She registered for the listserv — which she believes was very helpful because it 
provides a wider perspective on the issue and encourages collaboration and participation of LAs from 
all over the country. Although she does not typically have the opportunity to participate in the web 
chats, the LA does download the transcripts and shares the information learned with others. Further, 
the LA reported that she went to both the public and private sides of the EMSTAC web site very 
often. She expressed that the site was particularly useful when the district refocused on the issue of 
disproportionality because she was able to then use the site to pull information about the issue from 
the private side of the web site. This information she then disseminated among persons in the district 
in order to build support for addressing the problem of disproportionality. 

The LA is planning to continue implementing the 1ST model in the 17 schools next year. She 
expects the initiative will continue to be funded and is hoping that the school board will respond 
favorably to the newly collected outcome data that she recently presented by increasing the funding 
for the initiative so that the program can be expanded to new sites in the coming years. The program 
has been successful in addressing the issue of disproportionate representation in the pilot school sites 
and she is looking forward to continuing to positively impact the educational opportunities for all 
students, including students with disabilities. 
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EXHIBIT A-7: 

District 8 Technical Assistance 
Delivery Model, 2001-2002 
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District 9 



District Background 

A telephone interview was conducted with the Linking Agent (LA) of District 9, on April 1, 
2002. This particular school district, with a population of approximately 435,470, has 1 1 percent of 
the student body receiving special education services, 85.6 percent receiving free or reduced price 
lunch, and 90.4 percent are minority students. The Schools in District 9 are a Strategy II site. 

During the 2000-2001 school year, the LA focused on reducing referrals to special education 
through the School-Based Problem Solving intervention. The district has used this intervention for 
the past four years to solve the problem of the disproportionate representation of minority students in 
special education classes. The School-Based Problem Solving model, originally developed at 
Governor’s State University, is a comprehensive approach that includes the development of 
intervention plans with the goal of keeping students in the general education classroom. In addition, 
direct support is provided to the child, which the LA believes is a significant improvement compared 
to previous assistance models used in the past. 

In previous years the LA had collected, analyzed and synthesized data to gain a better 
understanding of disproportionate representation within the district. To address the issue of 
disproportionality, the LA had focused on consensus building to empower general and special 
educators to share accountability for all students. The LA met with the EMSTAC TA Liaison to 
discuss strategies for consensus building. This was necessary because historically, general educators 
viewed disproportionality as an issue affecting only special education teachers (and their students) 
but, in reality, this is an issue that affects everyone. For example, a major contributor for the high 
referral rates is believed to be instructional practices in general education programs. With a change in 
instructional practices, the general education teacher may be able to retain children in the classroom 
who may otherwise be referred for special education assessment. The LA decided to reach out to 
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administrators in general education to ask for their support in solving this problem. The LA has 
played a critical role in initiating progress in this area, from taking steps of consensus building to 
negotiating with special education administrators and reaching out to others through EMSTAC. 

Stage of Change 

Due to other initiatives taking a priority over disproportionality within the district, the LA has 
not been able to progress as much as she had hoped in addressing this issue. The other initiatives are 
supported by grants, include reading, medical support (assisting families who are not able to pay for 
medical support on their own), and testing (for students who are not able to participate in regular 
assessment). Currently, in addition to School Based Problem Solving, disproportionality is being 
addressed through assessment procedures, some of which are initiated by the state as a result of the 
reauthorization of IDEA. The district is examining its procedures to ensure that children with 
disabilities are included in general State and district-wide assessment programs, with appropriate 
accommodations and modifications. Additionally, when assessed for placement in special education 
programs, factors such as culture and language are taken into consideration when analyzing the 
assessment results. 

The LA had hoped to examine additional ways of helping schools work with principals, case 
managers, and teachers on this issue. There have been some positive changes in referral rates of 
students to special education classes due to the interventions and additional supports that are in place 
but the rates have risen in some areas as well. 

Important Factors 

One of the main factors that appears to be hindering the progress of the disproportionality 
initiative is the number of other initiatives the district is implementing. District 9 is a very large 
district that must deal with many issues simultaneously. As a result, priorities shift, and the reality is 
that every issue cannot receive the same level of attention at all times. The situation is further 
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compounded by the tremendous shortage of qualified general education teachers given the 
significance of good teaching for reducing referrals to special education. Other barriers include lack 
of time and space, in terms of sharing information with individuals on a more personal basis. 

On a more positive note, the LA feels that both the district and individual schools are open to 
change, and that most of the initiatives adopted within the past few years to improve outcomes for 
children have been effective. The district is trying to ensure that students with disabilities are “treated 
as regular kids,” and that less emphasis is placed on their disabilities. “We are always spinning our 
wheels on something, but we’re making a difference,” stated the LA during the interview. 

EMSTAC Supports 

Besides EMSTAC, the district collaborates with organizations such as the National Institute 
for Urban School Improvement, the Urban Collaborative of Greater City Schools, and various 
universities and colleges in the area. 

The LA, whose primary roles are analyzing policy and sharing information with a group of 
individuals who make decisions about future directions, expressed that her TA Liaison had been very 
helpful and had always provided information when it was needed. She appreciated the TA Liaison’s 
efforts in trying to initiate greater dialog with the district on the topic of disproportionality. Though 
the LA cited the physical distance between Washington DC and her district as a barrier, she benefited 
from the materials her TA Liaison sent to her, the chat events, as well as being able to share ideas 
with her TA Liaison, who has a strong research background. The LA has accessed both the public 
and private sections of the EMSTAC website, found the reading product particularly helpful, and 
used it for an in-service session at the district. 

Although the disproportionality initiative is facing some barriers at this time, the LA will 
continue to move forward with her plans to build consensus. She is searching for effective strategies 
to promote greater collaboration between the two disciplines. She has been and will continue to be 
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the catalyst for change throughout the district for disproportionality. The LA recognizes that 
addressing disproportionality requires a systemic approach that focuses on reductions of 
inappropriate referrals and improved achievement. 

She also believes that change is a slow process, but it will happen. Her motivation is fueled 
by her persistence to improve outcomes for all children. One important step that the district is taking 
is its expansion of the School-Based Problem Solving Approach into additional schools. Due to its 
success, the district has set a goal to implement this approach in every school. To sustain the progress 
that has been achieved, primarily in elementary schools, the district will place greater emphasis on 
procedures and instruction in middle schools. 

Recognizing the complexities inherent in disproportionality, the LA has plans to concentrate 
on improved instruction in special education classes, increased parental involvement, and other 
strategies to improve outcomes for students who may be misplaced with the goal of eventually 
returning these students to the mainstream. The LA believes that hers is a mammoth task, but she has 
inherited a problem that has been around for over 30 years and she is determined to make a 
difference. 
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EXHIBIT A-8: 

District 9 Technical Assistance 
Delivery Model, 2001-2002 
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District 10 



District Background 

District 10 is a small suburban district with a total enrollment for the in 2000-2001 of 
approximately 1,000 students among three school buildings, with 73.6 percent classified as low 
income and 24 percent receiving special education services. District 10 (Strategy II site) was 
recruited to work with EMSTAC by regional education service agency staff who attended the 1999 
Council for Exceptional Children (CEC) conference; after the educational service agency expressed 
an interest in working with EMSTAC, they contacted representatives from their member districts, 
and District 10 was among those districts that indicated interest in participating based upon a 
presentation by EMSTAC staff in October of 1999. District 10 staff received EMSTAC training in 
November of 1999. 

During a site visit to District 10 in November of 2001, it was reported that interest in working 
with EMSTAC stemmed from the possibility that EMSTAC might be useful in helping the district 
identify a reading program as required by the state empowerment act (citing persistent low test 
scores). Based upon his “limited understanding” of EMSTAC, the Superintendent for Curriculum 
and Instruction for District 10 agreed to have one of the teachers serve as an EMSTAC Linking 
Agent (LA). In addition to funds provided by the state to improve identified areas of need, it was 
believed that EMSTAC could be used to assist the LA in researching and selecting potential reading 
programs. 

The targeted area of need in District 10 has been reading improvement. Through a state 
mandate. District 10, along with other districts in the state, has been required to develop a plan for 
improvement and then identify and select programs (which are funded in part by $500,000 from the 
state) to begin implementing in classrooms. Reading was selected in District 10 due to the district’s 
low scores on the state’s School Assessment; test scores were comparatively lower in reading than in 
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math. During the 2000-2001 school year, the LA and the Assistant Superintendent worked to identify 
a reading program that would be useful in directly addressing reading skills measured on the state 
assessment. After sifting through the research and visiting other school districts a decision was made 
to implement Direct Instruction (DI). Reportedly, the selection process occurred in conjunction with 
significant input from teachers, including having them visit other schools using DI and having them 
talk with vendors. 

During the 2000-2001 school year, EMSTAC attempted to provide support and information 
to the LA; however, due to communication difficulties (lack of e-mail/intemet access and a change in 
the LA’s classroom duties) much of this information was not used. Further, the assistant 
superintendent who worked closely with the LA reported that he did not clearly understand 
EMSTAC’s role, particularly after EMSTAC indicated that it could not positively endorse any one 
reading program. Ultimately, however, information was provided by EMSTAC on DI, which likely 
assisted in helping the district make a decision about this program. By the end of the 2000-2001 
school year. District 10 had acquired the information and resources they needed to begin the 
implementation of the program during the following school year. 

Stage of Change 

After selecting Direct Instruction during the 2000-01 school year, teachers were trained in the 
summer of 2001 and began using the model at the start of the current (2001-02) year. It was noted 
that in contrast to many districts where DI is opposed by teachers due to its highly scripted nature, 
teachers in District 10 provided little resistance. The Assistant Superintendent speculated that this 
was because the state mandate made it very clear to everyone that something had to be done, and that 
continuing to do “business as usual” was not an option. 

District 10, then, is in the first year of actually using DI (“Try” Stage of the Havelock model), 
but is in the second year of the 3-year cycle in which it must improve performance or risk being 
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dissolved. It was reported that support for the teachers is largely being provided by a reading 
consultant who works for the district, in addition to outside support being provided by the regional 
education service agency and a local university. The trained LA has returned to the classroom and in 
his absence the Assistant Superintendent is largely performing the LA role. Through his efforts the 
district has continued to progress through the change cycle. 

Important Factors 

Similar to previous years, the leadership provided by the assistant superintendent has been 
critical to the district’s implementation of Direct Instruction. In addition, the assistant superintendent 
stressed the importance he placed on ensuring that school staff had input into selecting a program and 
developing a plan for implementation. He reported that collaboration with others (e.g. reading 
consultant, teachers, LA, and outside consultants) has been critical to their success to date. 

EMSTAC Support 

Initially the support provided by EMSTAC occurred directly as a result of the relationship 
between the EMSTAC TA Liaison, the TA Liaison at the regional education service agency and the 
trained LA. However, once the LA re-entered the classroom in the Fall of 2001, it became extremely 
difficult to regularly communicate with the LA as his job responsibilities shifted to the classroom. As 
a result, the direct support provided by EMSTAC has tapered off during the past year. Although the 
assistant superintendent has seamlessly replaced the LA, communication with him has been difficult 
due to his lack of clarity regarding how EMSTAC could assist the district with their chosen goals. 
This ambiguity regarding EMSTAC likely can be traced to the previous school year when the 
assistant superintendent had apparently hoped and expected that EMSTAC would be more directive 
in making recommendations regarding reading programs that could meet the district’s needs. When 
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this did not occur, the district pushed forward with their initiative but found it difficult to determine 
how EMSTAC could assist in their efforts. 
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EXHIBIT A-9: 

District 10 Technical Assistance 
Delivery Model, 2001-2002 
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District 11 



District Background 

EMSTAC conducted a phone interview with the Linking Agent (LA) from District 1 1 on 
May 9, 2002. District 1 1 has an enrollment of 55,660 students, with almost 12 percent of the students 
identified for special education services. Of the students receiving special education services between 
the ages of six to 21, approximately 46 percent are fully included in the general education classroom, 
22 percent use resource classrooms, another 31 percent are in self-contained classrooms, and one 
percent are either in a special school or hospital/homebound. Additionally, 62 percent of students 
receive free or reduced price lunch and 70 percent of students are minorities. 

District 1 Ibegan working with EMSTAC as a Strategy II district in the fall of 1999. 

EMSTAC was introduced to this district with the help of an external, professional contact at a local 
university. Prior to involvement with EMSTAC, the district identified the disproportionate 
overrepresentation of minority students in special education as an area needing to be addressed. 

During the initial year of collaboration between EMSTAC and District 1 1, the LA 
coordinated a formal presentation and workshop conducted by a researcher in the area of 
disproportionality. This workshop had two purposes: 1) to heighten awareness of minority 
achievement issues among the district’s Pupil Appraisal staff and 2) to pull together district leaders to 
begin to build consensus around the need for and relevance of increased minority achievement in the 
district. The LA continued to work with EMSTAC and this researcher to survey the literature and 
identify an intervention to meet the needs of the district in the area of disproportionality. Through 
this process, the LA concluded that the best way to stop inappropriate referrals was to provide 
appropriate instruction and interventions in general education classrooms. Soon thereafter, the district 
adopted PAM, the Pupil Appraisal Model, and began to collaborate with Louisiana State University 
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(LSU) for implementation support. The goals of the initiative are to prevent inappropriate referrals 
and to have students meet accountability standards. 

State of Change 

During the past year, the school district has enlarged its school improvement efforts to 
address the areas of transition, personnel preparation, alternative schools, and least restrictive 
environment. Although the district is focusing its attention to improve in these four areas, PAM has 
been maintained in the original schools. In addition. District 1 1 has identified 25 schools to begin 
implementing direct instruction as part of the least restrictive environment initiative, and the LA also 
plans to present PAM to these schools. The district is also using the Strategic Instructional Model 
(SIM), which is a trainer of trainers model, developed by the University of Kansas to address the 
issue of least restrictive environment. 

Additionally, the EMSTAC TA Liaison and the LA are in the planning stages of organizing a 
meeting with district administrators to discuss disproportionality in District 1 1 . The goal of this 
meeting is to build consensus about the need to continue to address disproportionality and the need to 
support the PAM initiative. Due to scheduling conflicts, this meeting has not yet occurred, but the 
LA and TA Liaison continue to discuss ways of building consensus within the district. 

District 1 1 is currently in the “Extend” stage and moving into the “Renew” stage of the 
Havelock model. They have already implemented PAM and are looking to incorporate it into a new 
initiative so that PAM can be extended to more schools. The district is also making efforts to 
examine disproportionality on an on-going basis. Thus, they are moving toward the “Renew” stage as 
they are examining how the PAM model can be used within new initiatives identified in the school 
district. While District 1 1 is in the advanced stages of the Havelock model for the PAM initiative, 
they are also in a stage of transition in which they are examining other areas of improvement for the 
district. 
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Important Factors 

There have been several factors contributing to the implementation and sustaining of the 
PAM initiative. The buy-in and support of the district administration has been extremely important. 
From the beginning, the LA organized presentations and a workshop to heighten awareness and build 
consensus among district administrators. The LA has a strong relationship with district leaders and 
has made the support and collaboration of district leadership an on-going priority. 

District 1 1 is currently examining several areas in which they will begin new school 
improvement initiatives. The introduction of these new initiatives also played an important role in the 
district’s priorities and how disproportionality fits into the district’s goals. While these other areas 
have become a priority in terms of district resources, the issue of disproportionality continues to be 
addressed. Further, the LA and the district continue to explore how the PAM initiative can be 
included within their efforts to address educating students in the least restrictive environment. 

EMSTAC Support 

The LA reported that information on the listserv and website have been very helpful forms of 
EMSTAC support. For example. District 1 1 recently began implementing new lEP software, and the 
LA was able to learn about other districts’ experiences and impressions of the program through 
communication on EMSTAC ’s listserv, the LA Exchange. The LA often shares this information with 
instructional specialists and has received positive feedback indicating that the resources were helpful. 

Over that past three years, EMSTAC has worked with District 1 1 on strategies and initiatives 
to address the overrepresentation of minority students referred for special education services. 
Throughout this process, the LA stated that EMSTAC has been influential in changing the 
philosophy and thinking about disproportionality within the district. 
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EXHIBIT A-10: 

District 11 Technical Assistance 
Delivery Model, 2001-2002 
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District 12 



District Background 

District 12 has 87 campuses divided among four regions, with a total district enrollment of 
57,636 students. Of the entire population, 95.7 percent are minorities, many of whom are at-risk (52 
percent) and/or English Language Learners (32 percent). Eighty-five percent of the district’s student 
population participates in the free or reduced price lunch program. Approximately 13 percent of the 
district’s students receive special education services. District 12 also includes a military fort, which 
serves as a separate site for military families some of whom have children with disabilities. 

District 12 was recruited to work with EMSTAC in 1999. At that time, EMSTAC was 
looking at a number of districts across the country that were demographically similar to already 
existing Strategy I districts; also, EMSTAC was interested in expanding its area of service to this 
region. A phone call was made to the director of special education to see whether the district would 
be interested in working with EMSTAC. After an EMSTAC presentation in District 12 that June, 
district officials decided to participate. 

In previous years District 12 has focused on a variety of initiatives including contending with 
a citation from the state for unusually high referral and placement rates of students in special 
education, improved reading and other literacy skills in students, increasing professional 
development opportunities for teachers, improving test scores of special education students, and 
providing services in the least restrictive environment. This year, due to a change of position by the 
Linking Agent (LA) and District 12’s narrowed focus, the continued implementation of a Balanced 
Literacy Initiative and Strategies for Academic Success have becomes the primary focuses. 
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Stage of Change 



The Balanced Literacy Initiative provides teachers with opportunities to learn sound teaching 
methodologies for reading instruction, acquire materials for the implementation of these practices in 
the classroom, and opportunities for mentorship through a literacy coach. This program was chosen 
to address the declining reading scores of students in special education. In its first year of existence, 
it was implemented exclusively in special education settings. Based on the success, the district has 
decided to scale up implementation to many general education settings. In these efforts, the LA has 
promoted the training opportunities, provided technical assistance to teachers, and worked with 
district administrators on the logistics and funding issues that have arisen. Therefore, District 12 is 
has entered the “Extend” stage of change relative to the Havelock model. 

As the initiative has been extended to general education, the LA has encountered a few 
obstacles. First, many people questioned the difference between general and special education 
services, if the same curriculum was being used. This was addressed through numerous discussions 
with those who were concerned that entailed explaining that the curriculum could be applied in a 
different manner depending on the setting. Thus, it was understood that although similar materials 
and curricula were used, teaching methodologies were what distinguishes them. Second, the source 
of funding for this initiative became an issue. It was no longer exclusively the responsibility of 
special education or general education. This problem was exasperated by the fear that special 
education funds were supplanting general education operations. 

Two high schools and two middle schools have implemented the Strategies for Academic 
Success (SAS) studies skills program in response to the citation. Additionally, the program serves as 
a means of preventing new students being identified for special education services. This class, 
offering meta-cognitive and study skills, was open to students receiving special education services, 
students at-risk, and the general population. Although this program has been well established in four 
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schools, the LA has worked diligently to identify new schools for implementation. In efforts to 
accomplish this, the LA had to compare and contrast the methodologies with another program 
existing in the District. Next, she identified schools that the SAS program was best suited. Several 
new schools will be implementing the program in the next academic year. Based on the Havelock 
model, it appears District 12 is in the “Extend” stage of change on this initiative. 

Important Factors 

A number of factors have been integral to District 12’s success in implementing the 
aforementioned initiatives: 1) district-level leadership, 2) building-level leadership, 3) internal 
collaboration, and 4) buy-in by general and special education staff 

Although the District 12 has undergone extensive change in district leadership, the LA 
continues to find direction and support from them. She has been able to secure continued funding for 
the implementation of these initiatives, has been promoted to continue her professional growth 
through activities such as conference attendance, and has developed a good rapport with colleagues 
working in general education. The school principals have been instrumental in supporting the 
implementation of the initiatives. Many are enthusiastic about implementing new programs and 
welcome the assistance of the LA. 

District 12’s LA has also been crucial in the successful implementation of both SAS and 
Balanced Literacy. She coordinates activities with the administrators, school board, teachers, and 
principals, playing a “go-getter” role to the administrators’ idea-generating one. The LA is well liked 
and well respected by individuals in various capacities throughout her district and, for this reason, 
she is effective at obtaining buy-in for different programs and initiatives. 
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EMSTAC Support 



The District 12 Technical Assistance (TA) Liaison describes her role as assisting the LA as 
her district moves through the change process. This entails listening to needs, providing input, and 
solving problems when critical issues arise. In addition, the TA Liaison provides feedback on the 
LA’s planned activities (e.g., how to design and collect data to evaluate the intervention) and helps 
steer the LA toward important resources. This support is primarily delivered by e-mail or telephone 
and by one site visit in June 2000. 

The LA reports that EMSTAC has been particularly helpful in assisting the district to become 
more data-driven as it evaluates what works. She also reports that the district’s work with EMSTAC 
has helped in scaling up programs and confronting barriers to change. 

The LA and District 12 have found great benefit from working with EMSTAC on the 
Balanced Literacy and SAS initiatives. Many are pleased with the outcomes and anticipate wide- 
scale implementation next year. As the needs decrease for the LA’s focus on these areas, her job 
focus will change next year. She will now also be responsible for the identification and 
implementation of technologies to aid students in learning and living more independently. She looks 
forward to working with EMSTAC on these new issues. 
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EXHIBIT A-11: 

District 12 Technical Assistance 
Delivery Model, 2001-2002 
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District 13 



District Background 

Two EMSTAC staff members conducted a telephone interview with the Linking Agent (LA) 
and an elementary school teacher in District 13 on April 23, 2002. District 13 has an enrollment of 
roughly 17,061 students with 10 percent of students receiving special education services. Eleven 
percent receive free or reduced price lunch and approximately six percent are minority students. 
EMSTAC began working with District 13 in the spring of 2000 when several EMSTAC staff 
members conducted an on site training on the change process with a group of teachers and 
administrators in the district. Since that time, the current LA, who is the director of special education 
in the district, has taken the lead in being the Liaison between EMSTAC and the district. 

In previous years, the county conducted a needs assessment, which indicated that the school 
district should focus on the inclusion of students with disabilities. To address this need, the county 
became involved in Project WINS, an initiative focusing on staff development to equip teachers to 
teach in inclusive settings in two schools in the district. Following the implementation of Project 
WINS, the state became interested in this initiative and desired to scale up the project to more 
schools. Other important partners included, the Governor’s Council, the State Board of Education, 
and Kennesaw State University who all initiated Project Winning Team, which would provide 
funding to achieve similar goals as Project WINS. This support enabled participating schools to hire 
an additional teacher, fund three teachers to get their masters in special education, and to provide 
school-wide professional development. 

Stage of Change 

District 13 continues to address the need of inclusion through Project Winning Team, and the 
project has expanded to include seven schools within the district. The LA is responsible for the 
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coordination of the program at the district level, and she recruited local schools to participate in the 
grant. The LA is currently training two new facilitators to help with Project Winning Team. 

Last year District 13 was in Havelock Stage 4, “Acquire,” and Havelock Stage 5, “Try.” The 
district was taking an expansive view of the chosen intervention and reaching out for new resources 
to contribute to the inclusion efforts. During the 2001-2002 school year. Project Winning Team 
moved into Stage 6, “Extend,” of the Havelock model. Currently the program has expanded to 
include seven participating schools and the LA indicates that the schools are progressively taking 
more ownership of the initiative. Each school develops a plan and goals that they will accomplish 
throughout the year and, primarily, school staff take the lead in implementing the program. The LA 
collaborates with schools to solve problems when they arise, and she is also training more people to 
facilitate and sustain this initiative. She reported that “schools actively request support when they 
need it, and we work with them to solve problems.” 

Important Factors 

Strong support from district level leadership and building-level leadership and the buy-in 
from teachers have been important factors in the progression of Project Winning Team. The district 
leadership, including the LA, has provided the schools with the opportunity to participate in the 
project as well as provided on-going support in the form of professional development activities and 
collaborative problem solving. In addition, the district has built relationships with the Kennesaw 
State University and other resources that have proved to be valuable in supporting the inclusion 
initiative in the district. 

The building-level leadership within the schools has also been imperative to the success and 
expansion of Project Winning Team. With the support of the district and Kennesaw State University, 
each school implements the program autonomously. The LA reports that the “principals and special 
education teachers have really run with the program.” The funding for project Winning Team will be 
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ending next year, but the LA believes that the inclusion strategies will continue because participating 
schools have made it part of their philosophy. 

Finally, the buy-in from the teachers has been another important factor. We interviewed a 
teacher from an elementary school implementing Project Winning Team, who indicated that there 
has been an attitude of acceptance within the school. She reported that “seeing children with 
disabilities with non-disabled peers enhances the teachers’ expectations for the children as well as the 
children’s expectations for themselves.’’ The teacher also expressed that the program has produced 
long lasting results, and she thinks the staff will want to continue with inclusion after the project 
ends. 

EMSTAC Support 

Throughout this year, each school has become more autonomous in implementing Project 
Winning Team. Therefore, the role of the LA in Project Winning Team has become that of support 
and training other facilitators to help with the program. The LA reports that she has spent less time 
communicating with EMSTAC this year than in previous years because the program has been 
running very smoothly. Additionally, the LA had been out sick for a long period this year, which 
limited participation in EMSTAC activities. She indicated that there were a few chat events that she 
wanted to participate in, but she was on leave during that time. The TA Liaison primarily supports 
the LA by touching basing with her and providing the encouragement to continue to engage in TA 
support activities at the local level. 
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EXHIBIT A-12: 

District 13 Technical Assistance 
Delivery Model, 2001-2002 
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District 14 



District Background 

District 14 serve 66,918 students in 77 schools. Of these students, 10.6 percent receive 
special education services, 52.9 percent are of racially and ethnically minority families, and 31.6 
percent participate in the free or reduced price lunch program. EMSTAC is working with two schools 
in the district through a district-based Linking Agent (LA). 

District 14 became involved with EMSTAC when the TA Liaison called them to share 
information about resources available through EMSTAC. District staff agreed that joining EMSTAC 
was positive and agreed to work with EMSTAC. The LA completed training in January 2001. Since 
then, the LA role shifted to a new LA. 

The LA is working with EMSTAC on two initiatives; collaborative teaching at one middle 
school and Teacher Support Specialist (TSS) mentoring program at one elementary school. Both 
initiatives were initiated by the district. The collaborative teaching initiative involves general and 
special education teachers teaming in general education classrooms. Cooperative teaching has been 
in place in District 14, but is new to the middle school. Regarding the TSS mentoring program, the 
state has had a mentoring certification program, but previously did not have a program for special 
education. The TSS mentoring initiative that District 14 is working with EMSTAC on is funded 
through a CEC-OSEP grant that fills the gap and provides a mentoring program for special educators. 
The grant was received around the same time as District 14 and EMSTAC initiated a relationship. 

District 14 is very open to using external technical assistance related to providing services for 
children with special needs. They are currently working on several other initiatives related to special 
education. These initiatives include a reading initiative that encourages closer work with general 
education; a behavior management program for severe, aggressive, and self-injurious behavior; the 
development of rubrics for teachers to see what they are implementing as part of the reading process; 
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support services for students with autism; and balance scorecard to improvement achievement and 
staff development. Additionally, the district has received a number of grants and is participating in 
national studies. In the past five years, the district has adopted differentiated instruction and added 
Parent Liaisons (parents of children with disabilities) with state funds to work to resolve 
disagreements. 

Additional resources come from Project WINS from Kennesaw State University. Project 
WINS, which is state-funded, has been a very helpful resource in working with schools on inclusion. 
They will work with the school as much as the school requests. Their assistance includes help in the 
areas of curriculum, accommodations, working together, training, teacher observation, and providing 
relief time for field trips. Additionally, they provided $5,000 that paid for training. Except for the 
CEC-OSEP grant for the mentoring program, the district did not secure outside sources of funding 
for these initiatives. 

Stage of Change 

The LA’s main goals for this year differ for each of the initiatives. The goal for the 
collaborative teaching initiative is to begin working in a school that had no previous experience with 
the initiative. Additionally, the LA would like to expand this initiative and link agencies in order to 
support the teachers in the middle school. For the TSS mentoring program, the LA aims to continue 
to gather useful information to provide to the mentees in order to retain new teachers and offer ideas 
for recmitment. 

TSS mentoring classes began in November 2001 for special education mentors; this was the 
first class being offered exclusively for special education teachers. Classes were offered weekly in 
the evenings and ended in May (2002). The special education mentor teacher that was interviewed 
was asked by the LA to be a mentor for a new special education staff member at his elementary 
school. It is his first time serving as a mentor and he has learned a lot from the training, including 
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working with others, pointing new staff members in new directions, effectively implementing 
inclusion, and working with special education forms and procedures. The special education mentor 
teacher meets with his mentee daily. He observes the mentee’s classroom and, together, they 
brainstorm strategies to modify behaviors, lesson plans, and other instructional objectives. The TSS 
mentoring program is working well, as the support has proven to be helpful. The only barriers faced 
were the late start of the TSS mentoring program and a lot of planning time. The special education 
mentor teacher found implementation ran smoothly thanks to the efforts of the LA. The LA initiated 
the mentor training classes for special education teachers, recommended the special education mentor 
teacher, and cleared the way with his principal. 

The cooperative teaching initiative is being implemented at a middle school. The principal 
has been extremely supportive and is a self-proclaimed “cheerleader” for the program and has been 
active in coordinating staff development and budgeting for the initiative. Eight teachers volunteered 
to pilot co-teaching in 2000-2001; they were the core teachers. Approximately 20 teachers were 
trained in the summer 2001 and 24 more in January 2002. As new teachers are hired, they are asked 
to attend the training. Forty-four of the 58 staff at the middle school have been trained. The training 
was co-sponsored by District 14 and Project WINNS. The principal found the support of Project 
WINS and the LA helpful. His contact with the LA was more frequent in the early stages of the 
initiative. 

In addition to EMSTAC, the middle school works with Project WINS on cooperative 
teaching. The LA finds Project WINS and the school to be a great team. The LA uses other sources 
of technical assistance, including a website that two professors have set up and a professor at Georgia 
State that has a NIH grant. The State Department of Education is used for technical assistance on 
some issues, including teacher certification. The cooperative teaching initiative has been very 
positive. The principal is very supportive of the initiative. Several teachers are providing cooperative 
teaching and the school plans to expand the program next year. 
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During the time that District 14 has been working with EMSTAC, they have advanced 
through Havelock’s process of change. When they initially came on board, they were in the “Care” 
and “Relate” stages. They had identified their needs and began to establish contacts and build 
relationships. The district has been able to progress through the “Examine,” “Acquire,” and “Try” 
stages. They spent little time in the “Examine” stage, and instead moved rapidly to the “Acquire” 
stage, where they accessed numerous resources, including those from EMSTAC. As they 
implemented interventions to address their needs, they moved into the “Try” stage. 

Important Factors 

District 14 has successfully implemented the two initiatives that they are working on with 
EMSTAC. They have had local supports that have been helpful with both initiatives. The LA 
indicated that if they had not had local supports, they would have used EMSTAC more. Each of the 
sites had strong building-level leadership that was supportive of the initiative. The principal at the 
middle school has been extremely supportive, as noted by the LA and himself At the elementary 
school, the special education teacher mentor felt supported by his principal and was thankful that he 
allowed him the time to leave his classroom to observe his mentee’s classroom. Both initiatives were 
both district-initiated and voluntary. This combination brought individuals who wanted to be part of 
the initiative in and provided local supports. 

For the collaborative teaching initiative, teacher buy-in was an influential factor in expanding 
the initiative. The principal has had positive prior experience with cooperative teaching and showed 
the teachers that it works. At a previous school, they moved from a segregated model to fully 
inclusive and showed improvements on test scores within three years. Additionally, the core teachers 
that piloted the initiative were influential when they discussed their experiences with cooperative 
teaching. The teachers bought into cooperative teaching because it was proven to work. The teachers 
have been receptive, and staff development has been positive. 
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EMSTAC Support 

The LA had a lot of contact with the TA Liaison initially. Once the initiatives were started, 
they have not had the need for as frequent contact and typically are in touch monthly. The TA 
Liaison offered to visit the site, but the LA declined as her schedule would not allow time for a visit. 
The most common interaction between the LA and TA Liaison is when the LA requests information 
from the TA Liaison. Other interactions are to touch base and provide updates. She has found 
EMSTAC and her TA Liaison helpful in identifying research-based materials and motivating her to 
keep the process moving. 

The LA has utilized many of EMSTAC ’s resources. She mainly uses the website, which she 
finds the most helpful of EMSTAC’s resources. She visits the public and private parts of the website, 
which she finds useful and useable. The LA believes it would be more useful for school-based 
individuals to access the website and recommends that EMSTAC offer access. 

The listserv has been helpful, although she does not use it as much as she has in the past. The 
LA had technical problems as she attempted to participate in one of EMSTAC’s chat events. She 
successfully participated in another chat event that she described as “okay.” The LA has not 
requested or been sent any written products from EMSTAC. The LA supports several programs and 
believes it would be useful for school-based individuals to connect to EMSTAC resources. 

In conclusion. District 14 has successfully implemented cooperative teaching and TSS 
mentoring, each in one school. The cooperative teaching initiative is doing well on its own at this 
stage, with the support of Project WINS. The middle school plans to expand with more teachers next 
year. The TSS mentoring initiative served its purpose for the new staff member at the elementary 
school; the mentor and mentee will continue an informal mentoring relationship next year. The LA 
hopes to train the mentor trainers to infuse special education into the mentoring program next year. 
While the EMSTAC support has been helpful, local supports have also proven to be beneficial. Had 
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these supports not been in place, EMSTAC’s role may have been more integral to the success of 
these initiatives. The LA mentioned that this could be an issue in the future due to budget concerns 
and the possibility of reducing some supports and staff development. Finally, the LA found that it 
would be helpful if school-based individuals could access EMSTAC resources, particularly the 
website. 
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EXHIBIT A-1 3: 

District 14 Technical Assistance 
Delivery Model, 2001-2002 
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District 15 



District Background 

A phone interview was conducted with the Linking Agent (LA) for District 15 (Strategy II 
site) on April 17, 2002. Of the district’s approximately 2,953 students, 10 percent receive special 
education services, 72 percent are eligible to receive free or reduced price lunch, and 78 percent are 
minority students. One unique feature of this district is that every Wednesday the district has a half- 
day; in-service activity for the afternoon, and the students receive full credit for the day. 

During the 2000-2001 academic year the “Alternative Classroom” (AC) program, a model 
incorporating techniques of a number of research-based behavioral interventions that provide 
intensive services to students with behavioral disabilities, was implemented in the district. Students 
are referred to the program if they have shown behavioral problems such as attacking fellow students 
or teachers or other forms of physical aggression and violence. Except for a few students who have 
spent time in mental hospitals, the great majority of students have not been formally identified as 
having emotional disturbance. 

The schedule during the day for the students who participate in the program is as follows: a 
half hour of physical education, two hours of counseling, two hours of academics, and then an hour 
for work with the behavior interventionist at the end of the day. Ideally, a student spends two or three 
months in the AC classroom, and then is able to be reintegrated successfully back into a general 
education classroom. 

Stage of Change 

The school district appears to be in the fifth stage, “Try,” of the Havelock model of change, 
following a recent change in LAs. Currently, the AC program is continuing to be implemented at one 
school in District 15, with both first and second grade students participating. There are eight students 
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in the self-contained AC program class, three of whom have been hospitalized in the past. This year 
there were some students with severe disabilities in the AC program, which made its implementation 
somewhat more difficult. 

Besides grades five and six using the “Child Study Form” that the AC Program developed, 
the program is only being used exclusively at one elementary school, where the LA works. However, 
the LA is in the process of trying to convince the upper grades in the elementary schools that the 
program can benefit their students. She has met with the school board to try and expand the program 
to other grades. 

Important Factors 

This year, in contrast to last year, the students in the AC Program were placed in a junior 
high school building and, according to the LA, were exposed to behaviors that could have been 
detrimental to their progress in learning appropriate behaviors. They ate lunch and had physical 
education classes alongside the junior high school students. 

Factors that may have affected the program in a positive manner this past year include having 
more square footage at the new site, having a full-time teaching assistant in the classroom, and 
having a security guard (that worked for the entire school). In addition, the LA felt that the district is 
very open to supporting this intervention. 

EMSTAC Supports 

The LA reported that she gets much help from EMSTAC in answering questions and is in 
contact with her TA Liaison on a weekly basis, and on occasion more often than that. She has gained 
much from the abundance of research and conference information that her TA Liaison has shared 
with her. In her words, “it’s been invaluable.” She has also benefited from the LA listserv and has 
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found the private side of the EMSTAC website useful. Due to technical difficulties with her 
computer, expressed by other LAs as well, the LA could not access the Chat Events. 

Although the program is working well as it stands now, the LA felt that it could be improved. 
In an ideal situation, there would be a AC facility on each campus, with teachers that would be 
trained in alternative ways of teaching, with assistants, financial supports, and family counseling. 
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EXHIBIT A-14: 

District 15 Technical Assistance Delivery Model, 2001-2002 
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DISTRICT 16 



District Background 

A phone interview was conducted with the Linking Agent (LA) for the District 16 on May 
22, 2002. District 16 is a small district serving five growing, middle-class suburbs. The district has 
approximately 3,369 students, of which 13.6 percent receive special education services, 6.3 percent 
are eligible to receive firee or reduced price lunch, and 2.9 percent are minority students. District 16 is 
a member of an education service agency, a regional support network operated through federal, state, 
and district funds, designed to help meet the needs of the 42 districts in their metropolitan area. Two 
EMSTAC Technical Assistance (TA) Liaisons work with two of the education service agency’s TA 
Liaisons, who are employed as staff development specialists for the agency. Originally, five districts 
with in the education service agency network participated in EMSTAC activities. Since EMSTAC ’s 
initial contact with these districts, some of them have terminated their involvement, and some new 
districts have begun work with EMSTAC. 

District 16, a Strategy II site, was recruited in 1999 through the education agency at the 
Council for Exceptional Children (CEC) annual conference. The LA was trained in January, 2000. 
Since the inception of its collaboration with EMSTAC, District 16 targeted area of need has been to 
improve students’ study skills at the middle school level. In previous years, the LA obtained school 
and district approval to implement a program to teach study skills to students with special needs in 
the middle school. She also created an action plan and completed a proposal review firom her doctoral 
advisor (due to her desire to use her data and findings for her doctorate). Her goals for last year were 
to select specific interventions, to identify teachers with whom to work, and to train teachers to use 
the interventions. According to these activities, the district appeared to be in the “Acquire” stage of 
Havelock’s change process with the intention of moving into the “Try” stage during last year’s 
evaluation. 
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Stage of Change 

Due to the LA being on maternity leave for four months this past year, she was unable to 
devote the time necessary in order for her to progress nearly as much as she would have liked to, and 
anticipated that she would. She came to the conclusion that her original plan, to create an entire study 
skills curriculum, was too grand to implement, so she decided to focus on a narrower topic with a 
smaller breadth and depth. The LA plans to talk with teachers about what study skills they think are 
appropriate for middle school students, and determine how many students use study skills. She 
remains in the “Acquire” stage of her efforts, and continues to plan for implementation. The LA 
reported that her next intention was to create a survey to find out what types of study skills teachers 
think students are ready for at the middle school level. 

Important Factors 

The district is always working on different projects, according to the LA, and they are 
relatively open to change. During this past year the district made efforts to focus on the population of 
students with emotional disturbance, and they hired a support staff person to help with providing 
assistance for those programs. They also created a new position of special education administrator, 
who is responsible for the special education staff, and hired someone full time for it. These resources 
allowed the LA more time and availability to focus on her EMSTAC efforts. With regard to the study 
skills initiative the school board approved, though the LA does not have outside sources of funding, 
she has access to a university library, which she has found helpful. 

Like last year’s evaluation, this year’s interview found that time was the largest barrier to 
providing technical assistance in the district. The LA is the only school psychologist in the district, so 
she is kept very busy. For example, if much testing needs to be done in the district, she must focus on 
that and drop her other efforts. She does work that no one else in the district can do, such as sitting on 
child screening committees, which by itself (without her other roles and responsibilities) requires one 
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week of work per month. Furthermore, because the LA was the sole EMSTAC representative within 
the district, any progress within the district was a direct result of her efforts. Therefore, when the LA 
went on maternity leave, progress with this initiative was placed on hiatus. 

EMSTAC Support 

More than anything else, the LA expressed gratitude for the eagerness that her TA Liaisons 
have shown for wanting to provide support, and that they always lead her in the right direction. 
EMSTAC has provided the LA with assistance with selecting an appropriate intervention for her 
district, culling through various research and other informational materials relating to the topic of 
study skills, and has communicated with the LA via e-mail and phone from time to time to offer 
assistance and to keep her abreast of EMSTAC products and activities. This past school year, 
EMSTAC assisted the LA with gathering sample instruments and other research information for her 
to develop the survey mentioned above. 

During the current school year, the LA has tried to move forward with the implementation of 
a research-based intervention in middle school classrooms. However, the demands of her job, having 
her dissertation and implementation efforts take a change of direction, and being on maternity leave 
have been important factors that have affected her EMSTAC involvement. Importantly, the lone LA 
effort in attempting to make this initiative occur has contributed to the lack of progress, as others in 
the district have not been involved in planning this initiative. The LA has taken each challenge in 
stride as she continues with her efforts. As she settles back into her job and balances major life 
adjustments, she anticipates completing her survey and administering it in the next school year. 
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EXHIBIT A-15: 

District 16 Technical Assistance 
Delivery Model, 2001-2002 
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District 17 



District Background 

On February 11, 2002, two EMSTAC evaluators traveled to District 17 to conduct an 
interview with the Linking Agent (LA) on the progress of her work in the district over the past year. 
District 17 is a Strategy III district that has been working with EMSTAC on the issue of 
disproportionate representation of minority students since 2000. District 17 has a current enrollment 
of 30,655 students, approximately 21,000 of whom are enrolled in elementary and middle schools. 
Minority students comprise approximately 51% of the total population and 53% of the students in 
District 17 participate in the free or reduced price lunch program. Almost 15% of students receive 
special education services. 

The district has been involved in a number of school reform efforts over the past several 
years. For example, an inclusion initiative led to the successful inclusion of all students with 
behavioral and emotional disorders. Although the initiative did meet some initial resistance by 
teachers, the union, and parents, the program was ultimately accepted and has proven to be effective. 
Further, the district has been conducting an experimental project using curriculum-based 
measurement as a reading strategy. This initiative uses experimental and control group schools and 
examines the impact of weekly probes that are designed to help students with reading problems and 
students with special needs. The district also provides crisis prevention training to administrators, 
teachers, and all related school personnel who work with students with disabilities. This training 
provides skills for de-escalating inappropriate behaviors and provides the skills to prevent a crisis. 

The relationship between EMSTAC and District 1 7 began when the LA attended a summer 
workshop at Harvard University. One of the focus groups that she attended was lead by an EMSTAC 
TA liaison. During the session and throughout the workshop, the TA Liaison and the LA discussed 
their shared interests and discussed the services that EMSTAC could offer to District 17 and the 
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work that the LA was facilitating in the school district. Once the TA Liaison had returned from the 
workshop she sent an EMSTAC welcome packet to the new potential LA. The LA immediately 
shared the information with her director who expressed enthusiasm for working with EMSTAC. The 
information was then shared with the assistant superintendent, who was not only exploring options 
for obtaining technical assistance services, but also was aware of EMSTAC and the work 
accomplished through the center. “It only took about an hour for him [the assistant superintendent] to 
talk to my [the LA] director and decide that we were going to utilize [EMSTAC] to help us with the 
disproportionality project. And it just came together like that.” (From February 11, 2002 interview) 
The LA contacted EMSTAC, identified the area of disproportionality as an area that District 17 
would like to address with assistance and support from EMSTAC and has been successfully working 
with members of the EMSTAC disproportionality team for the past year and one-half 

District 17 first became aware of a problem of disproportionality in the district in 1998 when 
a curriculum management audit was completed by Phi Delta Kappa. Based from this audit, it was 
found that many students, primarily Afiican- American students, were not meeting the academic 
expectations of the state. Phi Delta Kappa also examined the number of students who were being 
referred to or were receiving services in special education. These data reflected a disproportionate 
number of minority students both being referred to and served in special education programs. A 
community group was very concerned about the Phi Delta Kappa findings and brought the issue of 
disproportionality to the attention of the School Board and the Superintendent. In response, the LA 
was hired by the school district to address the issues of disproportionality. The LA currently works as 
the 504 Coordinator and the Home Instruction Coordinator for the district. She leads a longitudinal 
disproportionality study in the district that focuses on Literacy Development and Pre-Referral 
Intervention. 

During the first six months of working with EMSTAC, the LA and members of the 
disproportionality team conducted a meeting with the District 17 to help conceptualize avenues to 
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address the problem of disproportionate representation of minority students being referred for and 
served in special education and to prioritize the issues. Of the many issues discussed at this meeting, 
the team decided to focus efforts on the reduction of inappropriate referrals to special education and 
examine the current District 17 pre-referral process. The purpose of addressing this issue hinges on 
research that indicates a lack of consistent, culturally responsive, research-based pre-referral 
strategies may contribute to the current proliferation of minority students referred to special 
education. An informal survey was distributed by the special education director to teachers in the 
districts and found that at least five different iterations of a pre-referral program existed in APS. 
Further, the district did not have a way to evaluate which schools were doing an exemplary job in the 
pre-referral process and which schools were systematically failing their students. 

Stage of Change 

Over the past year, the LA has been busy gathering research about effective strategies and 
interventions designed to reduce inappropriate referrals of all students and establish a referral process 
that is research-based. The LA identified her primary goals as 1) collecting research on the issue of 
disproportionality and 2) reviewing the literature to ensure that she is up-to-date on all the issues 
surrounding disproportionality. In order to examine and address the problem of disproportionate 
representation of minority students in special education, the LA has designed a longitudinal study. 
Based on the collected research, guidance from the EMSTAC TA Liaison, and meetings with the 
director of special education and the superintendent. District 17 decided to conduct a study entitled 
“Creating Equitable Learning Communities for All Students.” The purpose of the study was to 
examine the procedures for referral to special education and the pre-referral strategies utilized by the 
district to prevent inappropriate referrals. 

The LA has also facilitated consensus-building activities among teachers, administrators and 
other related school staff in order to increase buy-in to address the issue of disproportionate 
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representation within the schools. She planned information and problem-solving meetings, delegated 
responsibilities, and overall, was the catalyst to ensure people were participating in the initiative. To 
organize all the information collected and make it accessible to all interested parties, the LA 
organized collected research into a binder that continues to grow and expand everyday. This 
information, combined with the collected baseline data, has built a strong foundation for work in the 
area of disproportionality. 

Currently, this district is in the Try stage of the Havelock model. Through working with 
EMSTAC, District 17 has combined the extant research with the data gathered in the district through 
the “Creating Equitable Learning Communities for All Students” instruments which were adapted to 
meet the particular needs in the district. They have begun to implement the study by scheduling 
interviews with the administrators and teachers in the district. 

Important Factors 

The collaborative and leadership skills of the LA were a tremendous factor contributing to 
the success in the district. The LA has an increased awareness of how to navigate networks in the 
school district and gamer support for ideas. She is able to delegate responsibilities, define the roles of 
team members, organize and facilitate meetings, and effectively collaborate with stakeholders to 
complete a district-wide study of prereferral interventions. It is her dynamic ability that serves as a 
catalyst for the actions that are implemented and sustained in the school district. 

In addition, district-wide leadership appeared to be a factor in the progress of District 17. In 
particular, the Director of Special Education has become a steady source of support and ensures that 
the necessary resources are provided for the LA and her team of stakeholders from the school system, 
a local university, and community members. In addition, the Director of Special Education has 
enlisted the support of additional high-level colleagues, including the Superintendent and Assistant 
Superintendent, to sustain the progress of the study this school year. 
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In addition to the factors above, the District has benefited from funding from various 
resources. More specifically, financial resources from the state and local school district have covered 
the expenses of supplies, materials, copying fees, and travel expenses. 

All of these factors were important to the continued collaborative efforts to conduct a district- 
wide study in the District 17. Continued support by district- wide leadership is needed to complete the 
project. Unfortunately, without the presence and support of a strong leadership, the strength of the 
collaborative efforts between EMSTAC and District 17 are uncertain, as well as the implementation 
of interventions based on the results of the district-wide study. In addition, it is necessary for the LA 
to continue her managerial responsibilities and duties to sustain progress. Finally, additional funding 
is also needed to sustain the study. Though resources and materials were funded for the 2001-2002 
academic year, the LA is participating in grant-writing activities to gamer additional financial 
support for the upcoming 2002-2003 school year. 

EMSTAC Support 

The LA indicated that the support received through her EMSTAC TA Liaison was cmcial to 
their progress and success in the study. Primarily, the LA utilized the TA Liaison for gathering and 
collecting research about disproportionality in order to build the LA’s knowledge base on the issue 
and to save the LA valuable time in searching for the appropriate materials/publications. The LA also 
communicated regularly with the EMSTAC TA Liaison for the purposes of touching base, updating 
the TA Liaison on new progress and activities, and providing strong encouragement and momentum 
for the LA to continue with the study regardless of how frustrated she felt at times. The LA said “the 
connection that EMSTAC provides between theory and practice is very good and very solid. I’ve 
taken things to people in other districts because of our collaboration with EMSTAC. We are now the 
model in the state for look at disproportionality.” 
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On average, the LA communicated with the EMSTAC TA Liaison two to three times a week. 
Modes of communication varied from telephone, to e-mail, to in-person visits. The LA also 
participated in on-line chat opportunities, which enabled her to interact and discuss her district’s area 
of concern with top expert researchers in the field. The LA believed the combination of the 
information provided through the EMSTAC website, the specific information gathered for the 
purposes of the study in District 17, and the in-person site visits to address issues separates EMSTAC 
from the majority of all other Technical Assistance programs. Finally, the LA commented that “What 
was most helpful to me, especially in the beginning, was the journaling. I could get on that thing and 
get everything out. And that really helped me, it was a catharsis for me.” 
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EXHIBIT A-16: 

District 17 Technical Assistance 
Delivery Model, 2001-2002 
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District 18 



District Background 

School District 18 enrolled a total of 493,678 pupils in the 2000-2001 school year. Of these, 
approximately 33 percent received free or reduced price lunch, 40 percent were minorities, and 15 
percent of students were identified for special education. This District has been working closely with 
Parents Inc., a statewide parent organization that has been partnering with AIR during the last year. 
Parents Inc. helped recruit them to participate in the EMSTAC training and technical assistance. 

The District, according to the Linking Agent (LA), has a substantial pool of talented people 
working in special education. The LA felt that this had been one of the most important supports for 
her work. The District also has a relationship with the University of Alaska at Anchorage. 
Furthermore, a variety of district personnel have been successful this year in obtaining grants for 
various projects. 

Stage of Change 

This year. District 18 is working to address a number of identified needs, including a new, 
state-mandated, online lEP program; functional behavioral assessments and behavior plans; a new 
teacher mentoring program; staff development; confidentiality; and standards and assessments. The 
LA is a secondary special education teacher consultant who primarily works to improve special 
education practice through professional development for teachers as well as parent involvement. She 
has been involved in many of the new interventions. 

In terms of the lEP program and the standards and assessments, the District appears to be at 
the “Acquire” stage of change in the Havelock cycle. Functional behavioral assessments, teacher 
mentoring and staff development appear to be at the “Try” stage. 
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Important Factors 



The LA felt that by far the most important factor in her District’s progress in implementing 
research-based change was the talent available within the District. “We have people power in the 
building,” she explained. “Many people in this office have been in education for 20 or more years, so 
there’s a lot of talent.” She reported that they communicate regularly through e-mail, in person, and 
by phone. They also have year-round staff development, including a summer academy for teachers to 
learn how to use technology in their practice. “The District,” she said, “gives people the information 
and provides the training at a lower cost. It’s very popular.” The importance of technology is 
reflected in the District’s mission statement, which states the goal of a 1:5 ratio of teachers to 
computers. This access to technology, as well as the District’s ample availability of technology 
support, will help teachers access and use the online lEPs. 

The biggest barriers District 1 8 has faced in implementing these initiatives have been a lack 
of time and money. This year, the LA felt that the limited number of staff prevented them from being 
able to accomplish all they had set out to do. Fortunately, her supervisor has been successful in 
obtaining additional resources, and they expect to be able to advance further in their goals next year. 

EMSTAC Support 

The staff development department of District 1 8 arranged for a small group of special 
education personnel, including the LA, to take the online EMSTAC training in the fall of 2001. She 
said the training “was interesting for me,” and especially helpful for those, like herself, who were in 
new positions. She found the material well presented, and she felt that the case studies had been 
particularly helpful to her in her staff development work. 

The group was given just a day and a half to complete the training because they were under a 
time cmnch. For this reason, the LA did not realize until late in the school year that EMSTAC 
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offered continuous support once the training had been completed. Due to her many responsibilities, 
she was unable to take the time to explore some of the additional means of technical assistance 
offered by her TA Liaison. “I realize now that it could have been a wealth of information for me, if I 
had accessed it,” she explained. “We are used to going with something tried and true. . .and I did not 
form the habit. I put it on the shelf” 

Another reason she said she did not take as much advantage of EMSTAC technical support 
as she could have was that “I felt like there was enough people support in the office so that I didn’t 
need EMSTAC support online. If there hadn’t been support internally, then I would have gone to 
look for additional outside support.” 

Perhaps most importantly, this LA observed that the personal touch would be very important 
for busy school district personnel like herself. “If I really wanted support, you would need a regional 
person here, visible, to keep reminding us that this is available. The first line of action is to go to a 
person who has had experience. It’s more efficient to go to a person than to go online.” Although she 
had received numerous e-mail reminders fi-om her Liaison, she felt that “if you don’t have a person to 
remind you... we forget that you’re there. It has to be a habit. To form a new habit, you have to have 
someone walk you through the next step.” She suggested that once someone finishes the training, he 
or she needs to receive reminders about the listserv, the chat, and the availability of research-based 
information on the web site. These reminders, she felt, should come from several sources at 
EMSTAC, not just the TA Liaison. “What [my TA Liaison] did [by sending lots of e-mail 
reminders], that’s very, very helpful. I give you permission to bombard me with e-mails!” 

Quite late in the school year, the LA discovered the many kinds of information and support 
available from EMSTAC. For example, through a telephone call with her Liaison, she found out that 
EMSTAC has a Study Skills product on the private side of the web page. She planned to download 
this information, which she said would be “exactly what we were looking for” for an upcoming staff 
development training. “Now that I have extra time [during the summer],” she said, “I want to get 
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online and see what you have.” The Liaison also reminded her about the availability of the listserv, 
which she said she planned to sign up for that same day. The LA felt that the online training had been 
quite helpful in her professional development work. She also appreciated the professional quality and 
ease of use of EMSTAC’s materials. 

Like so many district personnel, however, the LA held multiple responsibilities and was 
significantly limited in the time available for exploring the various means of support available to her. 
The reasons she stated for her limited use of EMSTAC supports were: 1) lack of knowledge 
regarding what was available; 2) lack of time; and 3) not feeling the urgency to change her current 
habits for seeking support. Once the school year came to a close and she had time to give some 
attention to communications from her TA Liaison, the LA realized what she had been missing, and 
began making plans to take better advantage of the support offered to her. 

Some of the lessons that might be gleaned from this case include; 

1. Once LAs complete training. Liaisons should make immediate contact, preferably 
through a personal phone call (if a physical visit is not feasible) to touch base and 
discuss future steps. 

2. Information packets and reminder e-mails should be considered important follow ups, 
but perhaps not the primary means of initial communication subsequent to the 
completion of training. 

3. Although an LA fails to respond to initial attempts to make contact, TA Liaisons 
should not assume a lack of interest. In many cases, as in the case of District 18, LAs 
may be too busy to read e-mails or respond to phone calls, and thus may not realize 
the supports available to them. At least in this case, persistence paid off, and the LA 
reiterated that she did not mind at all receiving multiple reminders. 

Now that the LA has had the opportunity to access some of the many services and resources 

EMSTAC has to offer, she is more likely to begin the habit of using these resources in her work 

during the coming school year. 
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EXHIBIT A-17: 

District 18 Technical Assistance 
Delivery Model, 2001-2002 



TA Liaison 




Exhibit Key 










Indicates a strong relationship 






Indicates a moderate relationship 
Indicates an existing relationship 




w 







BESTCOPY AVAILABLE 



A-98 



:69 



American Institutes for Research 



District 19 



District Background 

District 19 is the fifth largest school district in its state. It serves 74,491 students in 1 17 
schools. The District serves 28,763 elementary school students in 77 schools. Of the elementary 
school students in the District, 13.5 percent receive special education services and 26 percent receive 
free or reduced price lunch. EMSTAC is currently working with one elementary school in the district 
on the topic of disproportionality. The school serves 315 students, of whom 16.5 percent receive 
special education services and 23 percent receive free or reduced price lunch. The Linking Agent 
(LA) is the principal of the school; she stated that there is a great deal of diversity in the school. The 
backgrounds of the students “range from affluent to poverty” and she seeks to build on the diversity 
and ensure achievement for all students. 

District 19 was introduced to EMSTAC at the State Disproportionality Conference where two 
TA Liaisons presented information. They caught the attention of the Associate Superintendent of 
Instruction, who asked them to come back to help with the Leadership Conference. The LA got 
involved in 2000 when she was the assistant principal of an elementary school. She met her TA 
Liaison and participated in the online training that same year. 

The LA was transferred from the assistant principal role at one elementary school to serve as 
principal at another elementary school in July 2001 . A replacement LA at her former school has 
started the online training. The transfer slowed the process down, but the LA is planning for next 
year (2002-2003). Since transferring, the LA has met with her TA Liaisons in order to define the 
purposes of the disproportionality project at her new school. Due to her new role as principal, she 
may look for a replacement LA at her new school. 

The LA worked on other initiatives while at the previous school. She was trained in 
Instructional Consultation. Additionally, the school used the minority achievement sections of the 
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manual that the TA Liaison helped them with. She found it difficult to implement the new system, in 
terms of merging it with the goals of the county. The LA has found it to be helpful if the intervention 
is aligned with the county, with room for variation as each school has different needs. She plans to 
try Fast Forward, a computerized phonemic awareness program, at her new school. 

Stage of Change 

The LA’s goal for the next school year is to work on issues of disproportionality and ensure 
achievement for all students. As part of the Annapolis Challenge Grant, she plans to base her work 
on school-level data. The LA plans to look for ways to increase minority student achievement in the 
context of special education issues. She plans to identify interventions that can be used before 
students begin the special education process. The LA plans to send a team from her school to the next 
Disproportionality Conference. 

The topic of focus, disproportionality, was chosen prior to EMSTAC involvement. The LA 
and the Director of Special Education for District 19 serve as Co-Chairs of the District 19 
Disproportionality Committee, which has been in place for three years. They wanted to meet with 
EMSTAC in order to get help in reducing the problem of disproportionality in the county. 

The topic of focus was originated by the Superintendent, who has since retired. The issue of 
disproportionality came to the forefront when some State counties were being required by the Office 
of Civil Rights to address the issue. At that stage, the District decided to look at their percentages 
and found that they, too, had a disproportionality problem, so they volunteered to work on this issue. 

During the time that the District has been working with EMSTAC, they have advanced 
through the Havelock process of change. When they initially began, they were in the “Care” stage. 
They had identified the need to address issues of disproportionality and ensure achievement for all 
students. The District has been able to progress through the “Relate” stage and is presently in the 
“Examine” stage. In the “Relate” stage, they established contacts and built relationships within the 
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system. They are now in the “Examine” stage, where they are planning for the upcoming school year 
by establishing objectives and searching for potential solutions. 

According to the LA, time has been the greatest barrier in getting this initiative up and 
mnning. She often finds that she does not have the time needed to access research. An additional 
barrier is the difficulty in “spreading the word;” While many acknowledge the problem, they do not 
know what to do about it. The LA finds her greatest resource is well-trained teachers: teachers who 
know alternative methods of instruction delivery and interventions (e.g., Lindamood or co-teaching). 
She appreciates good teachers who adapt to individualized student needs. She would like for the 
District to make information regarding disproportionality available through in-service. She would 
like EMSTAC’s help in order to learn about potential interventions and disseminate information. 

Important Factors 

The LA’s transfer from assistant principal at one school to principal at another slowed the 
process somewhat. She no longer has the time available in her new role to access research, spread the 
word, and move the initiative forward. Her greatest resource has been well-trained teachers. The 
motivation for this project is internal and the personal investment in ensuring that all students 
succeed, and disproportionality is reduced, benefits the project. 

EMSTAC Support 

The LA sees an added benefit in EMSTAC’s wealth of resources and information. She 
communicates with her TA Liaison approximately once a month. It is common for the TA Liaison to 
request data from the LA or for the LA to request information from the TA Liaison. Although this 
District is a Strategy III site, its close proximity has allowed for site visits throughout the year. The 
TA Liaison had made three site visits to the District this school year (2001-2002). 
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While the LA has utilized many of EMSTAC’s resources, she finds her TA Liaisons to be the 
most helpful aspect of EMSTAC’s support. She has found EMSTAC’s support in identifying 
research-based materials and help in coordinating TA events to be very helpful. She also believes 
that the encouragement provided by her TA Liaisons has helped her to move the process along. 

The LA has utilized the public and private sides of the website. She found both to provide 
useful and useable information. The LA found the listserv helpful, particularly the input from others, 
and information sharing. She found the chat events beneficial. However, she has two problems with 
the chat events: (a) the technical delay and (b) the way that they are set up. She thinks that the guest 
speakers are cut off by the continuous questions. The speakers have a lot of useful information and 
have so much input to offer, but it seems that their train of thought is interrupted because of all of the 
questions being asked. She recommends that the guest speaker present without interruption for the 
first half of the session and then have time for questions and answers. 

Due to her role as principal, the LA has been advised to find a replacement LA. She would 
like to maintain contact with TA Liaisons and use EMSTAC resources. She wants her teachers to 
eventually be able to access EMSTAC resources. At the county level, the LA would like to get 
involved with EMSTAC’s teleconferences as a means of linking with other school districts. She 
stated that it would be nice to be a part of it and would like to have the school faculty involved, as 
well. The LA stated that EMSTAC has been very helpful whenever she has asked for help. 

The LA hopes to find a replacement and send a team from her school to the State 
Disproportionality Conference as she plans to move forward on the initiative in 2002-2003. 
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EXHIBIT A-18: 

District 19 Technical Assistance 
Delivery Model, 2001-2002 
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District 20 



District Background 

District 20 enrolls approximately 18,000 students in 32 schools. It has 20 elementary schools 
and five middle schools and employs 1,350 fully certified teachers, of which 48 percent have a 
Master’s degree. Approximately 2,400 students receive special education services in the District, 
which represents 1 3 percent of its enrollment. Sixty percent of these students are in general education 
classes for 90 percent of the day. Reading assistance is the most common service; help in math is 
second. Twenty-five percent of the students in this District are eligible for the free or reduced price 
lunch program. 

The EMSTAC Linking Agent (LA) is a seasoned teaching professional with over 25 years of 
experience in education. The majority of her experience has been as a teacher in general and special 
education classrooms. She has taught a variety of students including students with learning 
disabilities, mental disabilities, and severe disabilities including autism. The LA’s experience has 
been predominantly at the elementary grade levels. The LA came to District 20 as a Curriculum 
Facilitator for Special Services approximately one and a half years ago. 

New in her position as a Curriculum Facilitator, the LA attended an information session at 
the 2001 Council for Exceptional Children Convention held in Vancouver, British Columbia, 

Canada. The session was led by EMSTAC staff, provided attendees with an overview of EMSTAC, 
and offered audience members an opportunity to get involved with EMSTAC. The role of an LA and 
the online training were described. During the session, the LA explained that she was new in her 
position as a Curriculum Facilitator for District 20 and was seeking resources that would help her in 
her new position. 
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EMSTAC staff followed up with her after the CEC convention and, shortly thereafter, she 
began the EMSTAC online training. The LA completed the training in June 2001 and was assigned 
to the EMSTAC Accessing the General Education Curriculum and Inclusion Team. 

The LA is currently the leader of a district-wide task force to develop an integrated, cohesive 
curriculum for students with mental disabilities. The ultimate goal of the Curriculum Task Force was 
to implement a curriculum that spanned grade levels and schools within District 20 and that would 
result in improved post-school outcomes for students with mental disabilities. 

The District Central office is eager to improve instruction to facilitate student outcomes 
through the implementation of new interventions and programs. District persoimel take advantage of 
opportunities at the state and regional levels to participate in training and staff development. The LA 
receives ongoing support from district administrators to participate in new training and development 
programs. The District uses district-wide, interdisciplinary groups or task forces to address specific 
needs and provides them with financial and informational support. 

Educators, including the LA, receive support through programs offered by the state’s 
Department of Education. District personnel also participate on state facilitated task forces such as a 
recent one developed regarding alternate assessment. The State-level Alternate Assessment Task 
Force receives technical assistance from the U.S. Department of Education’s North Central Regional 
Educational Lab (NCREL) through materials and face-to-face training. District 20 personnel take 
advantage of resources provided by the State Education Agency, the State’s technical assistance 
regional agency. Additionally, administrators from the District use available state funds to support 
District task forces, such as the Curriculum Task Force, for which the LA serves as the leader. 

The funds that the District has allocated to the curriculum initiative have enabled the Task 
Force to make recommendations regarding the purchase of new programs such as the Saxon Math 
Program, the Life Centered Career Education Program, and the health-related. Everyday Life Skills 
Curriculum. With these new interventions. District personnel received training and support from the 
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state agencies described above, as well as from product developers affiliated with these specific 
programs. 

Stage of Change 

When the LA assumed her position as a Curriculum Facilitator, there were two predominant 
needs within the District. The first need related to identifying and implementing a K-12 curriculum 
for students with mental disabilities (MD) across grade levels and across schools within the District. 
The impetus for this need came from the realization by District personnel that 50 percent of students 
with disabilities who left secondary schools in the District were unable to sustain gainful 
employment. The need for a coordinated curriculum based on validated research was apparent, and 
the interdisciplinary district task force described above was convened to address this topic. The LA 
assumed leadership of this Task Force shortly before her partnership with EMSTAC. 

A second need identified by District 20 pertained to the identification of procedures and best 
practices for alternate assessment. Hence, the support that the LA receives from the State Department 
of Education, a local educational agency, and EMSTAC are important to facilitate her ability to 
address this need. The LA attends state-facilitated professional development events, including those 
activities administered by the regional educational agency to acquire information regarding this topic. 

The District’s Need Related to An Integrated Curriculum for Students with 
Mental Disabilities 

The work of the LA and the District Task Force has been successful in implementing an 
integrated curriculum for students with mental disabilities. The multi-grade curriculum consists of 
the following widely used programs: 

1. Life Centered Career Education (Council for Exceptional Children (CEC). This is a transition 
curriculum developed by IDEA researcher, Donn Brolin, that includes instruction targeted at 
developing daily living skills, personal social skills, and occupational guidance and 
exploration. The curriculum is appropriate for students in the upper elementary school 
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through adults, including individuals with mild mental disabilities, learning disabilities, 
limited English proficiency, or who are at risk. 

2. Saxon Math Program at the elementary level (Saxon Publishers). Script-driven curriculum 
with emphasis on skill development. Program uses practice, workbooks, posters, lesson 
plans, and weekly quizzes. 

3. Handwriting without Tears (Olsen). A developmentally based handwriting program that can 
be taught individually or in small groups. The program is multisensory in nature whose 
purpose is to promote mastery and fluency of handwriting. 

4. Health Curriculum (not specific to any one developer). The District pulled aspects of various 
curricula together to develop this health initiative. 

5. Everyday Life Skills (American Guidance System). Health curriculum targeted toward 
students in grades 9-12 that helps students to make successful transitions after high school 
through the development of functional skills. 

District 20 is in the pilot phase of implementing these programs. This corresponds to 
Havelock’s fifth stage of the change process, “Try.” In the pilot phase, classroom personnel have 
received training and guidance regarding the implementation of these programs and have begun to 
implement the curriculum in their classrooms. Task Force members, with the supervision and 
leadership of the LA, support classroom personnel and assist to collect information regarding the 
success of the interventions. 

Task Force members continue to collect information regarding the impact and outcomes of 
these interventions during this pilot phase. One way in which the programs are evaluated is through 
interviews and qualitative data received from those teaching professionals who are implementing the 
interventions. The LA recounted feedback from one teaching professional, “I never thought this kid 
could do this,” affirming both the positive impact that the integrated curriculum has had upon student 
achievement and, importantly, the raised expectations for student performance that are expressed by 
educators. 
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The District’s Need Related to Developing Alternate Assessment 
Procedures 

The LA serves on a state-level Task Force to address this topic. The District’s state is the 
only one in the United States that does not have state-level standards regarding student curriculum 
and learning. Therefore, local jurisdictions are responsible for developing their own standards and the 
assessment measures to determine whether students are meeting these standards. The LA is working 
with state level colleagues and district personnel to develop local standards and assessment tools that 
are aligned with these standards. 

The LA stated that the discretion and autonomy that educators in local districts have in 
developing standards and assessment criteria provides them with a sense of ownership over the 
process. Regarding this particular need, the district is in the Havelock “Examine” stage. District 20 is 
continuing to collect resources and assess the viability of various processes related to the alternate 
assessment program. 

Important Factors 

The LA has strong experience in a variety of educational settings that provides her with 
knowledge and skills to facilitate her role as a leader and someone to be trusted by district 
colleagues. She presents herself as an enthusiastic and credible purveyor of information and support. 
Through her leadership positions she is able to impart her expertise to improve practices related to 
both integrating a curriculum for students with mental disabilities and to develop consistent and 
equitable processes for alternate assessment. She receives support from District administrators 
through their agreement to fund specific initiatives and is given the autonomy to make decisions 
regarding curriculum and assessment programs. The District’s use of task forces to address needs 
facilitates collaboration among various professionals across buildings and across disciplines. 

Although building-level staff are sometimes slow to respond to change, when building-level 
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professionals observe the positive impact of interventions, such as the curriculum interventions, this 
is helpful in enhancing the willingness of resistant personnel to accept these new programs. 

The open communication that exists within District 20 appears to positively influence the 
acquisition of resources and the implementation of new programs. The LA expressed a strong desire 
for continuous improvement and an interest in always wanting to make things better. This passion is 
reflected in her voluntary participation in EMSTAC and her continued interest in acquiring external 
resources. 

EMSTAC Support 

The EMSTAC TA Liaison has been especially helpful to the LA and the District. Through 
contact approximately three times a month, the Liaison shares materials with the LA that contain 
examples of alternate assessment procedures in place in other local districts. This information is 
useful, since having information about practical solutions is important. The LA has received 
numerous hard copy documents from her Liaison that she maintains in an organized system. She uses 
all of the resources, especially those generated through the EMSTAC Listserv, LA Exchange, for her 
own learning and also to disseminate to her colleagues. For example, when a learning disabilities 
resource was recently posted on the EMSTAC Listserv, the LA forwarded this resource to a 
colleague who was addressing this topic. The LA’s colleague then utilized the resource to help 
parents understand the topic at an lEP meeting. The LA serves as a conduit of information for her 
peers in District 20. Her position as a Curriculum Facilitator provides her with the mobility to 
interact with a variety of school professionals and to exchange and share resources. The LA 
described the support and resources she receives from EMSTAC to be extremely important to her 
role. 

The support that the LA receives from internal and external sources, including EMSTAC, is 
strong. The intrinsic personal qualities of the LA, such as leadership and experience, combined with 
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the structural and administrative processes in place in the District, positively impact upon this 
District’s ability to identify needs, gather resources, and implement research-based programs. 
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EXHIBIT A-19: 

District 20 Technical Assistance 
Delivery Model, 2001-2002 
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District 21 



District Background 

District 21 located in a growing urban area, serves nearly 23,000 students in grades 
kindergarten through 12. New developments and industry have contributed to a student population 
increase of over 130 percent during the past 10 years. The district is comprised of 21 elementary, five 
middle, and four high schools. The student population is ethnically diverse: 61 percent white, 30 
percent Hispanic, five percent African-American, three percent Asian, and two percent Native 
American. 

The Linking Agent (LA) learned about EMSTAC through information he received while 
attending the Council for Exceptional Children (CEC) convention in Vancouver in April 2000. He 
began working with EMSTAC in the fall of 2000. In the fall of2001, the LA transferred to District 
21. Upon starting his new job there, as Title I Coordinator, he proposed to his new administration 
that a continued relationship between him and EMSTAC be maintained. 

Before the LA came to District 21, several schools tried to implement Success for All with 
varied success. For example, one school has successfully implemented the program with the 
leadership and support of the principal and teachers. Another school used Success for All with 
modification to the curriculum that better fit the school’s needs. A third school has faced opposition 
to the program from teachers and could not get adequate buy-in for the initiative. 

District 21 has four primary information resources available to them, including: EMSTAC, 
West Ed Technical Assistance Center, the State Department of Education, and the University of 
Oregoii. EMSTAC has been used as a source of research-based information. West Ed has worked 
with the District on developing an action plan that will help the schools to implement changes to 
meet the requirements of the newly established state standards and to achieve better results for all 
children. The State Department of Education assists the school in implementing state standards and 
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ensuring state and federal compliance. Lastly, the University of Oregon has provided information on 
intervention implementation through technical assistance and conducting in-service training for 
teachers. 

Stage of Change 

After the LA left his position at Salt River Pima School District as Special Education 
Coordinator and joined District 21 as the Title I Coordinator, he found himself in a position that was 
conducive to implementing change. It was apparent to him shortly after taking the new position that 
District staff has a sincere desire to improve results for all children, but a clear direction or action 
plan needs to be developed. To accomplish this, two general goals have been identified: 1) help staff 
District-wide see the need for change and facilitate such change, and 2) promote staff buy-in to 
change through empowering staff throughout the change process. 

At this time, the District is in the “Relate” and “Examine” stages of the Havelock change 
model. The LA has distributed one form of a needs assessment to staff throughout the District and 
would like to conduct a supplementary needs assessment based on the nominal group technique 
presented in the EMSTAC training. The staff members have provided the LA with positive feedback 
on the process thus far. The data have not yet been analyzed on the first needs assessment, nor have 
definitive plans been made on the second format. The LA would like to use EMSTAC as a resource 
in selecting research-based practices to address the identified needs of the District. 

Important Factors 

District support stems from many streams, including external technical assistance, internal 
support, and holistic community buy-in. External technical assistance comes from WestEd, the State 
Department of Education, University of Oregon, and EMSTAC. The LA relies on these organizations 
for information and research-based practices especially in the areas of reading and behavior. He has 
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more heavily relied on these resources in his previous position, but plans to utilize them in the 
District. 

The LA identified many internal supports available to him and other District staff members. 
He noted that the Instructional Resource Center (IRC), an on-line information site, offers teachers: 

(1) Relevant phone numbers, (2) Staff orientation materials, (3) Internal promotion materials, on-line 
technical tutorials (e.g., instructions on how to make a class web page), (4) Access to content area 
specialists who are staffed at the IRC, (5) Opportunities for staff to become involved in District 
projects (e.g., how to have an Intel volunteer work in your classroom), and (6) Other staff 
development opportunities. The LA believes this offers staff members direct access to innovative, 
research-based teaching methodologies in a convenient, easily-accessible manner. 

Other internal supports include his supervisor’s support for his personal and professional 
development. This support includes flexibility in schedule, financial support, and allowing the LA 
time to research ideas on the Internet, attend conferences, work with EMSTAC, and read professional 
journals. This support is in line with his supervisor’s continued commitment to infusing research- 
based practices into the curriculum. 

Lastly, the LA identified that the community buy-in and support of initiatives has been very 
motivating. For example. District personnel, parents and students were actively involved in a recent 
needs assessment. Subsequently, the LA received much positive feedback about the process and 
outcomes of this assessment, indicating that all involved felt that those able to implement change 
heard their concerns and suggestions. Results will be used to direct future District initiatives. 

EMSTAC Support 

The LA relies on EMSTAC in identifying research-based practices. He has been able to do 
this in the past, and plans to continue to use it for this purpose in the District to address the goals 
identified from the District needs assessment. He participated in the chat events, received information 
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from his TA Liaison, and accessed information presented on both the public and private side of the 
website and through the listserv. Although he found all the information useful, he noted that the links 
to research-based information on the listserv were particularly helpful. The LA often saved resources 
and links from the website and listserv and established a library of information that he can pull from 
in his daily activities. 

EMSTAC has had a significant influence on the District 21 needs assessment process. 
Although this activity was required by the state, the LA indicated that he was much more zealous 
about completing it knowing the importance of acquiring input from a variety of stakeholders as part 
of implementing meaningful and lasting change. 
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EXHIBIT A-20: 

District 21 Technical Assistance 
Delivery Model, 2001-2002 
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District 22 



District Bacl^round 

District 22 is a Strategy III district that has been with EMSTAC for approximately one year. 
Since the District has not yet implemented a specific intervention program with EMSTAC, classroom 
observations or teacher interviews were not conducted. 

District 22 is the 23"* largest school district in the nation. Data provided by the District 
indicate a total student enrollment of 102,013 students. The students’ ethnic breakdown is 42 percent 
Afncan American, 47 percent white, four percent Asian, four percent Hispanic, and fewer than one 
percent American Indian/Multi-Racial. In the District, 81 Native languages are spoken and more than 
6,000 students are identified as English language learners. 

The District has identified the disproportionate representation of minority students in special 
education as a targeted area for change. It is seeking interventions for the disproportionality problem, 
which is primarily emanating from referrals for behavior issues. Due to their continued emphasis on 
the acquisition of relevant materials, resources, and strategies, the District is currently functioning at 
the “Acquire” stage of the Havelock change process. 

In addition to focusing on disproportionality, the District is currently working on several 
improvement initiatives that began last school year. One major initiative is an extensive review of the 
different methodologies for working with children with autism. Essentially, the District will increase 
its emphasis on the identification and implementation of research-based practices. The District is also 
involved in a major reading initiative that includes training in appropriate strategies and 
methodologies for all regular and special education teachers. To supplement this professional 
development initiative, the Exceptional Children’s Department provides additional support to special 
education teachers through Cecil Mercer’s “You Fly” training. This District has adopted the Open 
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Court reading program at the elementary level and will expand its focus on reading to establish 
District-wide reading programs at the middle school and high school levels. 

Another major initiative is the Balanced Score Card. Through this initiative, the previous 
superintendent established District-wide goals that aim to improve instruction for all children and 
developed service integration teams that include staff members from special education, regular 
education, and instructional support teams. To improve special education services, the Exceptional 
Children’s Department has established two specific goals: 1) to increase the number of children who 
exit special education programs at the end of grades 5 and 8; and 2) to increase the number of special 
education students who graduate with a regular diploma. 

District 22 has worked with the University of North Carolina at Charlotte (UNCC) on several 
collaborative grants. Currently, the District is partnering with UNCC on a grant to increase transition 
opportunities for special education students. Another collaborative grant between the District and 
UNCC aims to improve instruction by examining the results of portfolio assessments of children with 
cognitive difficulties. The District is also collaborating with the local chapter of the Association for 
Retarded Citizens. 

The Linking Agent (LA) first learned about EMSTAC through her coordinating director, who 
has since been promoted to assistant superintendent. In addition, the District’s deputy superintendent 
(now superintendent) was approached at a national conference about collaborating with EMSTAC. 

He agreed and referred EMSTAC to a District administrator to establish EMSTAC’s initial entry into 
the district. 

Stage Of Change 

The District has been grappling with issue of disproportionate representation of minorities in 
special education. As a result, disproportionality was identified as an area of need for the District to 
address. After meeting with EMSTAC staff, the director of special education decided that the 
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behavioral specialist would complement the team and asked her to complete the LA training also. 

She has become the lead person and has been involved in numerous discussions with the EMSTAC 
Technical Assistance (TA) Liaison to identify viable interventions for addressing disproportionality 
in the District. The behavioral specialist brings a strong background in formative and summative 
assessment to the effort. Her goal has been to work with EMSTAC to utilize her expertise to begin a 
District-wide effort to promote greater use of curriculum based assessment for early identification 
and monitoring of student needs and progress. 

The LA has also expressed interest in acquiring information about interventions that have 
been successful in other districts. In that vein, she is seeking greater collaboration with other LAs 
through telephone conference calls to discuss strategies for reducing disproportionate representation. 
Finally, to further define the District’s needs, the LA coordinates efforts to acquire pertinent 
statistical information about factors related to disproportionality and data for continuous monitoring. 

EMSTAC Support 

The LA and the TA Liaison initially explored a number of possible interventions to address 
disproportionate representation. After learning of an intervention that focuses on cohorts of students 
who are at risk for placement in special education, the LA expressed interest in implementing such an 
intervention. At one time, the LA had asked the EMSTAC TA Liaison to co-author a proposal to 
District superiors advising them of this intervention and asking their approval to implement it in 
selected schools throughout the district. 

Although the LA has not yet taken advantage of the EMSTAC chat sessions, she views the 
chats as one of the most beneficial aspects of working with EMSTAC. Moreover, the demands of her 
position have not permitted her to participate in many EMSTAC outreach activities, but she is still 
interested in getting in touch with people from other large districts to discuss common issues and 
viable solutions. The LA also believes that the LA training was a helpful reminder of the change 
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process. She believes that it is essential for people in leadership positions to recognize that change is 
inevitable. The LA downloaded and disseminated to her staff several documents from the EMSTAC 
training website. She has used this material to conduct training sessions for them. 

The LA would like EMSTAC’s support as the District prepares to implement a group 
intervention strategy to address the area of disproportionality in the district. Most importantly, the 
LA has increased district awareness of the problem, but believes that she will need the support of 
more people in her district to address it. The LA identified 29 coordinating teachers as potential 
change agents to help her implement interventions in the schools. The LA has involved them in 
different aspects of her plan and vision for addressing disproportionate representation in District 22. 
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EXHIBIT A-21: 

District 22 Technical Assistance 
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District 23 



District Background 

District 23 is composed mostly of suburban and urban neighborhoods and is the sixth largest 
school district in the country. During the 2001-2002 school year, the District had a total enrollment 
of 244,000 with approximately 46 percent minority students. Of the 46 percent, 29 percent are 
Hispanic, 14 percent are African American, seven percent are Asian, and one percent is Native 
American. Special education services are currently being provided to 1 1 percent of the student 
population and approximately 37 percent of students in the District participate in the free or reduced 
price lunch program. 

EMSTAC became involved with District 23 in the fall of 2000 when its Assistant 
Superintendent met the EMSTAC Director at an OSEP conference and brought information 
regarding EMSTAC to the District’s Director of Elementary School-Based Special Education 
Programs. The director completed the Strategy III training and then identified 10 individuals to serve 
as Linking Agents (LAs). Most of the LAs began the EMSTAC training in January 2001; however, 
to date, only four individuals have completed the training and are actively working with EMSTAC. 
This is due, in part, to the reorganization of District 23 into five regions. On April 18, 2002, 
EMSTAC staff members visited with four District LAs. The director was not available to participate 
in this meeting. The following descriptive summary reflects information gathered during the 
interview. 

As stated in last year’s evaluation report, the LAs were comprised of 10 Special Education 
Administrative Specialists (SEAS). Due to the reorganization of the district, the LAs no longer have 
the same job titles and are working in a different capacity than they have in the past. 
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Stage of Change 

District 23 is currently focusing on four, predetermined initiatives including (1) A+, (2) 
Inclusion, (3) Autism, and (4) Literacy. In this section we will describe the current status of each 
initiative and detail any plans for expansion. Each of the four initiatives and other District-wide and 
school-based priorities are based on the Matrix data that are comprised of student demographics, 
academic outcomes, and placement data. 

A+ Initiative 

This District-wide initiative seeks to improve accountability, access, and achievement. These 
three pillars are used as a backdrop to all district-wide and school-based activities and are considered 
critical in ensuring that the District meets the needs of all students. For example, when a region 
wanted to implement a new Positive Behavior Support Program in its schools, the region was 
required to develop a plan explicitly stating how this program would impact upon the three pillars. 
Additionally, the District uses this terminology as a means of communicating and promoting its 
success to the community at large. 

The District plans to continue to use the A+ initiative as an overriding theme in all district- 
wide activities. Under the A+ umbrella, new initiatives are being developed in some regions 
including a Behavior Redirection program. Palm Computing, and teachers as facilitators of change. 
The behavior redirection initiative aims to increase the integration of children with emotional 
disturbance into general education classrooms. The District has selected the FOCUS Program to 
address this goal. Educators and administrators feel that the program has the capacity to positively 
impact children not identified as having serious emotional/behavior disorders, but in need of more 
intense behavior management techniques. The Palm Computing program utilizes Palm hand held 
computers to enable teachers to differentiate instruction allowing a wider array of students to access 
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the general education curriculum. Mindspring developed the Palm program. Currently, this is only in 
use in a few classrooms, but there are plans to scale up to school-wide implementation next year. 
Lastly, the EMSTAC LAs have provided training to all teacher facilitators, one representing each of 
the District’s schools, on Havelock’s change model presented in the EMSTAC training. Although 
this training met with much resistance from the staff, the LAs attributed it to the lack of incentives 
offered to the participants (many of whom are not paid for this extra work) rather than their actual 
interest in the topic. Each of the programs included in the A+ initiative appears to be in the “Try” 
stage of the Havelock model. 

Inclusion Initiative 

This initiative aims to increase the level of inclusion of students with disabilities in general 
education classrooms. The goal is for all students with disabilities to have increased opportunities in 
general education settings. During the first year (2000-01), many teachers were extremely resistant to 
this initiative; however, this year (2001-02), it appears that as teachers’ awareness increases, their 
resistance decreases. Increased awareness can be attributed to numerous professional development 
opportunities outlining the benefits of inclusion as well as successful teaching techniques that can be 
used in classrooms with diverse student populations. 

The District plans to continue to conduct in-service training on this topic, and target schools 
that have a disproportionate number of students in self-contained classrooms as compared to other 
schools in the District. The Inclusion initiative is in the “Extend” stage of the Havelock change 
process. 

Autism Initiative 

The autism initiative has been developed through work with Roy Boss from the Autism 
Partnership, an organization in California. The goal of this collaborative effort is to improve the 
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teaching strategies of those working directly with students with autism. This program was developed 
as a variation of the discrete trial methodologies and rather than being applied in a one-on-one 
setting, this program is being used in a group setting. Additionally, many of the principles of Applied 
Behavior Analysis have been infused into this curriculum. Teacher training, ongoing support, and 
teacher-to-teacher mentoring are all key components of this program. 

All teachers working directly with students with autism were invited to participate in a two- 
day training outlining general characteristics of students with autism and teaching methodologies 
used with this population. Next, a select group of teachers was invited to participate in an intensive 
four-day training that incorporated hands-on teaching experiences in the application of discrete trial 
teaching and behavior modification methodologies. While the teachers participate in this portion of 
the training, another trainer works in the teacher’s classroom to rearrange the environment and 
prepare the students and classroom for the implementation of new methodologies. After these trained 
teachers return to their classrooms and begin to implement the program, their class is considered a 
model demonstration site for others. On a monthly basis, trainers from the Autism Partnership meet 
with model teachers to ensure that implementation is going well and provide support for problems 
encountered. Although the District is pleased with initial successes, they have found that some 
students are developing splinter skills that cannot be generalized to other environments. 

The District plans to continue to work with the Autism Partnership but would like to see the 
school develop the capacity to sustain the initiative with less dependence on this outside 
organization. Over the next school year. District 23 would like to develop and implement a plan to 
strengthen their model sites, deliver additional training, and support existing and new teachers 
working with students with autism. This places the autism initiative in the “Extend” stage of the 
Havelock model. 
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Literacy Initiative 

The Literacy Initiative is aimed at improving reading and writing skills of students with 
disabilities and includes the implementation of three programs: (1) Read 180, (2) Project Life, and 
(3) Project Stars. The Matrix data indicated that the largest reading and writing deficits were in the 
middle school grades. In previous years, this initiative has only been implemented at select schools 
within the District, but this year it has been extended to all middle schools, and a few additional 
elementary and high schools. Beyond a few technological barriers, the staff indicates that they are 
pleased with the program and feel that it benefits children. To substantiate that, data indicate that 
classes served have made academic gains of up to three grade levels within one academic year. This 
is significantly higher than the progress made by those not in the program. 

The District plans to continue the implementation of the Literacy Initiative and extend it 
district-wide within the next two years. 

Important Factors 

District level leadership appears to an important factor in the progress of several initiatives 
(A+, Autism Initiative, Literacy Initiative). In particular, the A-i- program has been put in place 
district-wide to ensure that all schools focus on the importance of achievement, accountability, and 
access for all children. 

Another factor that has contributed to the progress in District 23 is the strong internal 
collaboration among the LAs, teacher facilitators, outside resources, and school and District 
administrators. At this point last year, the LAs were uncertain how the reorganization of the district 
would affect services for students. It appears that the LAs have made a smooth transition into their 
new positions and have developed positive relationships with key individuals within their respective 
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regions. In addition, the LAs are able to utilize their relationship with EMSTAC to provide teachers 
with timely information and resources relevant to specific needs. 

In addition to the factors above, District 23 has benefited from financial resources provided 
by the District. These resources include money to support current school-based initiatives as well as 
district-wide initiatives. Each year the region coordinator has to provide a proposal to the Board of 
Education in order to gain approval of the next year’s funding that will sustain the current initiatives. 
At the time of the interview, no funding had been allocated or dispersed. 

All of the above factors will impact the continued growth and success of current and future 
initiatives within the District. Continued and increased support is needed at the District level in order 
to extend the programs to additional schools and provide training to teacher facilitators. On-going 
collaboration among district administrators, LAs, and teacher facilitators will help to ensure that 
classroom teachers are learning about and implementing research-based practices. The LAs stated 
that, “increasing the number of support staff in the District would allow more time for working with 
EMSTAC.” They also expressed an interest in having additional people from each region trained as 
EMSTAC LAs. 

EMSTAC Support 

The LAs have received support from EMSTAC in understanding the change process, 
facilitating in-service training, and in the reorganization of their district. Not all of the LAs have 
participated in the EMSTAC listserv, but the ones who have indicated that, “the topics are timely and 
interesting.” The LAs have made use of the EMSTAC funding product and the autism product in the 
development of staff development training. They also found the private side of the web site and the 
information contained in the training modules helpful. Also they reported that they maintain 
communication with EMSTAC at least six times per year either through e-mail or phone 
conversations. The majority of the communication has been to request information from the TA 
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Liaison and to provide data to the TA Liaison. In addition to these information exchanges, the LAs 
also communicate with their TA Liaison to provide progress updates on an “as needed” basis. 
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EXHIBIT A-22: 

District 23 Technical Assistance 
Delivery Model, 2001-2002 
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District 24 



District Background 

District 24 is located in a small rural community. The District was initially recruited for 
EMSTAC involvement at the 2001 Council for Exceptional Children Convention and has maintained 
a strong relationship ever since. The District has three elementary schools, a middle school, and a 
high school. During the 2001-2002 school year, the student population was 1,374. The ethnicity of 
the student population is homogeneous, with over 99 percent of the students being Caucasian. 
Slightly more than 10 percent of the students receive special education services. In addition, about 
seven percent of the students receive free or reduced price lunch. 

District 24 has over 100 certified staff members with a support staff of 75. Forty-five percent 
of the certified staff members have graduate degrees. The teacher to student ratio is 1 : 1 7 at the 
elementary level and 1:20 at the secondary level. 

The EMSTAC Linking Agent (LA) serves the district in a variety of capacities, such as a 
school-based literacy coach, a district-wide special education advisory group member, and a member 
of the district transition steering committee. She also works closely with the Northwest Board of 
Cooperative Educational Services (BOCES). As an LA, she has been instrumental in supporting the 
District in its continued efforts to develop and implement initiatives by providing research-based 
information on commercially available materials and teaching strategies, and disseminating such 
information through formal technical assistance activities or informal means to her colleagues and 
administration. 

Previously, the District had focused on several initiatives, including: 

1 . Assessment, 

2. Gang Resistance and Avoidance Training (GRAT), 

3. Life-Skills Program, and 
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Literacy. 



The assessment initiative was aimed at using assessment to direct instructional content, facilitate the 
ability of school personnel to diagnose student deficiencies, and build a foundation to discuss data 
based assessments with students’ families. The second initiative, GRAT, is a program administered 
by the Sheriff s office, which teaches students gang avoidance strategies. The Life Skills Program 
was geared towards students with developmental delays. It focused on skills such as doing the 
laundry, cooking, grocery shopping, leisure activities, and job skills. Lastly, the literacy initiative, 
based on the “train the trainers” model, aimed to integrate literacy skill development into the 
curriculum of all content areas. 

District 24 continues to receive support firom two sources: EMSTAC and the BOCES. 
EMSTAC has had the opportunity to provide technical assistance to the District through contact with 
the LA. Support activities have included providing research-based information on literacy teaching 
approaches and specific methodologies, and commercial materials. Also, the LA has taken advantage 
of information provided by EMSTAC on the website, through chat events, and the listserv. The 
District has also participated in three EMSTAC videoconferences on literacy instruction. 
Additionally, the BOCES has provided District 24 with materials, technical assistance, and external 
funding opportunities. Extensive libraries of teaching and professional development materials are 
available to educators. Technical assistance is also provided to the District through the LA 
networking with BOCES staff and in-service training provided at the schools. Lastly, this District 
was able to collaborate with the BOCES for a Lighthouse grant, which funded the training and 
stipends given to Literacy Coaches. 

Stage of Change 
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Currently, District 24 is still engaged in several initiatives to improve outcomes for children, 
with emphasis placed on: 1) continuance of the literacy initiative; and 2) developing the School-to- 
Career Exploration Program. 

The District has trained and utilized literacy coaches in the implementation of research-based 
teaching methodologies, identified and purchased new materials, integrated literacy skill 
development into all academic areas, and commenced a reading club for teachers. Each school in the 
District has identified at least two teachers to be literacy coaches. After receiving extensive training, 
the Literacy Coaches have been able to support their colleagues in the implementation of literacy 
instruction in all content areas, learning new teaching techniques at bi-monthly in-service training, 
and modeling teaching techniques. Additionally, Literacy Coaches identified new materials for 
acquisition. Their decision was based on students’ needs, research-based best practices, and 
effectiveness data. Lastly, the Literacy Coaches have established a book club for teachers. The book 
club is based on a metacognitive approach to understanding reading comprehension strategies. The 
teachers collaboratively select and read a book, utilize strategies that are currently being 
implemented in the classrooms, and discuss the book and their experience with each of the strategies. 

Based on the CSAP scores, the literacy initiative has had a positive impact on students’ skills. 
The percentage of students who are proficient in reading and mathematics increased, fi-om 70 percent 
in 1999 to 75 percent in 2001 and 50 percent in 2000 to 66 percent in 2001, respectively. 
Additionally, writing scores have maintained at just fewer than 50 percent. Based on the success of 
the literacy initiative, the knowledge that there is room for growth, and the strong commitment to 
literacy skill development. District 24 began funding the literacy initiative as Lighthouse Grant 
monies ceased. 

Previously, the District has only had a transition curriculum in place at the high school level. 
Recently, a need was identified for earlier emphasis on transition; thus, the School-to-Career 
Exploration Program was established. As part of this program, teachers will: 
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• Familiarize themselves with the District identified workplace competencies, 

• Integrate career awareness and exploration into their existing curriculums, 

• Develop a resource library to facilitate such awareness and exploration, and 

• Work closely with a district transition specialist. 

Although these goals have been established, the LA, along with the District transition steering 
committee, will conduct an in-depth needs assessment, develop an elementary and middle school 
School-to-Career Exploration Program curriculum, and author three handbooks for teachers, 
students, and the community. 

Mapping District progress in the change process requires examining the initiatives separately. 
According to the Havelock change model, the District’s literacy initiative appears to be in the 
“Extend” stage. Through the support of the Literacy Coaches, acquired materials, and in-service 
trainings, literacy instruction has become a priority for the District. More teachers are being trained 
in methodologies in literacy and are continually supported by the literacy coaches. The District is 
evaluating the effects of the literacy initiative on the students, and they are making adjustments to 
fine-tune the program as it expands. Additionally, the District has further committed to this initiative 
by supplementing the funding as the initial grant is coming to an end. 

The elementary and middle school School-to-Career Exploration Program is in the 
“Examine” stage of the Havelock cycle. They are exploring this need in more depth through a 
steering committee. Currently, they are in the process of conducting a needs assessment to examine 
how the program can best meet the needs of the students. 

Important Factors 

Collaboration of district administrators, school administrators and teachers, internal support 
systems and training, and flexibility of the programs have been integral factors in contributing to the 
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progress of the initiatives. In both initiatives, the district and school administrators have supported 
the programs and, along with the LA, have been the impetus for their implementation. For example, 
the principal of the middle school participates in the reading group with other teachers to show her 
support and to further understand the program that teachers are implementing. This buy-in and 
support of administrators has enhanced the openness of the teachers and community members to 
participate in the initiatives. Additionally, the internal support of the Literacy Team, the LA and the 
in-service training have been important factors. Teachers, especially those without prior experience 
in literacy instruction, have felt more comfortable in trying new interventions because of this internal 
support. For example, the LA has gone into classrooms to teach a lesson so that teachers can see an 
intervention modeled for them. Finally, the flexibility to modify programs based on teachers’ and 
students’ needs has been important for staffbuy-in and the success of the initiatives. The principal 
expressed that she solicits input from teachers about the status of the program, and they make 
adaptations when problems are identified. 

EMSTAC Support 

The LA continues to use EMSTAC as a resource for research-based information for these and 
other district initiatives. The LA stated that “receiving information about new programs that are 
being implemented across the country keeps me thinking outside of the box and also prevents me 
from feeling isolated.” The LA also expressed that as a rural district, their participation in the video 
conferencing events with EMSTAC has provided them with expert information on focused topics 
that has been extremely beneficial. The LA also reports that EMSTAC has been helpful in providing 
her with research-based information that she then disseminated to her colleagues. 
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EXHIBIT A-23: 

District 24 Technical Assistance 
Delivery Model, 2001-2002 
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District 25 



District Background 

On April 26, 2002 two EMSTAC staff conducted a telephone interview with the Linking 
Agent (LA) from District 25, who also serves as a Staff Development Specialist. District 25 is a large 
urban school district enrolling approximately 22,466 students. Approximately 94 percent of their 
student population is comprised of minority groups. Around 17.4 percent of the student body are 
children with disabilities and 68.8 percent are eligible for free or reduced price lunch. 

In the recent past, the District was taken over by an independent organization. Experiencing 
difficulties running a diverse and challenging school system resulted in them pulling out after half a 
year. At this point. District 25 was taken over by the State Department of Education. A Board of 
Trustees appointed by the state chose the new school superintendent. EMSTAC was recommended 
by the state and the relationship was formalized following a meeting with the new District Director 
of Special Education. According to the Technical Assistance (TA) Liaison, the support from the State 
Department of Education representatives was crucial to getting the district on board. It is hoped that 
working with EMSTAC will help link research and resources to the district. 

District 25 is going through a tremendous restructuring. In addition, the LA finds that neither 
organizational nor financial support is readily available as yet. Hartford has, however, received a 
grant for helping students with behavioral problems. To that effect, the LA sees her role as helping to 
bring information regarding programs and interventions regarding behavioral referrals into the 
district to implement change. While they have yet to adopt a formal program or initiative, the LA is 
hoping that it will materialize by the fall. 
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Stage of Change 



As mentioned above, Hartford will be the recipient of a grant related to reducing behavioral 
problems. Because of this, the LA’s work with EMSTAC and with the TA Liaison has focused 
primarily on obtaining resources and information that identifies viable alternatives to suspension and 
expulsion for children with disabilities. The LA describes, “we are still ironing things out before 
formally implementing anything.” According to the TA Liaison, the LA has been involved in 
delivering in-service training to district personnel as a result of working with EMSTAC. In addition, 
the LA is disseminating resources provided to them through EMSTAC to their staff, and indicated 
that the informational tools were indeed helpful to their district. For instance, the LA shared 
EMSTAC resources regarding alternatives to expulsion and suspension with both her administrative 
and teacher colleagues. She indicated, to her TA Liaison, that these resources would help 
practitioners consider other ways of reacting to student behavioral problems. 

When the District started working with EMSTAC, they were already cognizant of the 
challenges they had to overcome. They were able to draw on the necessary connections both within 
and outside their system to relate to their common goal of improving and enriching the school 
experiences of children with disabilities. The District is now at Stage 3 of Havelock’s change cycle 
model: they are “Examining” their specific situation more closely. They have also begun Stage 4 of 
Havelock’s cycle, as they work with EMSTAC on “Acquiring” both informational and financial 
resources. 

Important Factors 

Several factors are important to understanding District 25’s progress. First and foremost, in 
the words of the LA, the structural and organizational changes that the district is currently 
experiencing have played a major role in creating an “atmosphere of change.” The other factors 
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include the support received from both the school superintendent and the Director of Special 
Education; the commitment of the LA to the staff development position and to being the LA; the 
resources available on site at the Special Education Resource Library; and the resources and easy 
access to EMSTAC staff. 

Both interviews with the LA and with the TA Liaison stressed the significance of the recent 
changes within the District. While there is still considerable disorganization and a lack of funds for 
special education, there is a renewed sense of hope. This optimism is based on the support and 
leadership received from building-level administrators. Both the LA and the TA Liaison feel that the 
introduction of a new Special Education Director and a new school superintendent are indicative of 
the school board’s and the State Department of Education’s commitment to the District. The LA cites 
both the superintendent and the Special Education Director as being extremely supportive not only of 
each other, but also of new initiatives within special education. 

In addition, the LA’s full-time position as a Staff Development Specialist is also relatively 
new and exemplifies the district’s focus on strengthening their current system. Because the LA has a 
background as a practitioner, she is able to relate to and understand the needs of teachers in the 
District. The TA Liaison finds that the LA has shown a great interest in gathering information and 
resources that, in turn, facilitated her completion of the EMSTAC LA training and her interest in 
maintaining a partnership with EMSTAC. As mentioned previously, the LA sees her goal as a 
clearinghouse of sorts — to collect and disseminate information about relevant programs and 
interventions that will reduce the number of behavioral referrals and provide some ideas about viable 
alternatives. She is taking her time to “build a file on a range of topics” before jumping into any 
program. 

These efforts to collect and evaluate information are areas where EMSTAC has been most 
helpful. In fact, both the LA and the TA Liaison see the District’s relationship with EMSTAC as 
being highly beneficial in terms of the kinds of resources that are now readily available for 
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application. The LA has found both the quality of EMSTAC products and the support received from 
her TA Liaison as particularly helpful. 

Aside from EMSTAC, the LA mentions that the Special Education Resource Center that is 
located on-site is a great source of up-to-date information on special education. She utilizes it for 
researching program materials and tests, and for audio-visual materials. 

A possible challenge to any progress in the District is the resistance of teachers to change. 

The LA explains, “they have been neglected. . . for so long they were not part of the system.” The 
constant turnover with Principals and Vice Principals as well as the restructuring on the district level 
has left them feeling fhistrated, cynical, and wary of change. The LA recognizes that any effort to 
implement change at a district level must involve and be supported at the local level — by the 
practitioners in schools. Developing teacher buy-in will continue to be a priority for this district. 

EMSTAC Support 

An important factor for understanding the District’s progress has been their relationship with 
EMSTAC. On one level it symbolizes the State Department of Education’s involvement and 
commitment to improving the lives of children with disabilities. To the LA, it demonstrates that 
building-level leadership — such as the Director of Special Education and the school superintendent — 
are fully supportive not only of her role as a Staff Development Specialist, but also as an LA. On 
another level EMSTAC has been a new source of support, resources, and technical assistance in a 
troubled district. 

Initially the LA was unsure of what her role as an LA meant. But she is becoming far more 
comfortable as she develops an independent relationship with her TA Liaison. In the beginning, the 
LA felt that she did not understand the connections between EMSTAC, the US Department of 
Education, the State Department of Education, and District 25. For her, it spelled additional pressure 
locally, statewide, and Federally to make changes. However, she is now far more appreciative of 
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EMSTAC, and “loves the quick access to information.” Furthermore, her TA Liaison “knows exactly 
what I am looking for. She sends me materials, and asks if I have any needs or questions for her.” 
Thus far, EMSTAC has been very helpful in identifying research-based materials, facilitating support 
and follow-up, and providing motivation. For the LA, the most beneficial aspect of EMSTAC has 
been the accessibility to “top-notch resources” such as the product on direct instruction. 

The model for technical assistance in the District is slowly evolving as the LA gets more 
comfortable with her new roles — as LA and Staff Development Specialist. There is a strong impetus 
for change, both district-wide and state-wide, as the District attempts to restructure its public school 
system. Working in collaboration with EMSTAC and the TA Liaison, the LA is making every effort 
to collect resources in order to implement a program this coming fall to reduce student behavioral 
referrals. 
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EXHIBIT A-24: 

District 25 Technical Assistance 
Delivery Model, 2001-2002 
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District 26 



District Background 

District 26 is an urban district with 133,336 students. Of these students, 10.8 percent receive 
special education services, 89.7 percent are minorities, and 70.9 percent qualify for free or reduced 
price lunch. The elementary school where the Linking Agent (LA) works is in an historic African 
American community in the inner city of District 26 that has developed into a self-contained village. 
Within this elementary school, 99.6 percent of students are African American, 91.2 percent receive 
free or reduced price lunch and 17.2 percent have been special education tested. 

The school has been working with the 100 Black Men of America, Inc. since 1998. Last year, 
one of the mentors approached the Parent and Community Liaison at the elementary school involved 
with the 100 Black Men of America, Inc. and asked her if she would like to work on the 
disproportionate overrepresentation of African American males through the Wimberly Initiative that 
had been established between the 100 Black Men of America, Inc. and the National Association of 
State Directors of Special Education (NASDSE) in 1 999. Not knowing much about 
disproportionality, she did some research and talked to a few people associated with EMSTAC and 
100 Black Men of America, Inc. and decided to get involved. She then went through the LA training 
and has been working with EMSTAC in conjunction with the 100 Black Men of America, Inc. on 
issues relating to disproportionality since fall 2001. 

Stage of Change 

Reading has been the focus in the district for multiple years. Under the former 
superintendent, the district was mandated to select one of a number of approved teaching models that 
required reading as a focus. This year, however, with the entrance of a new superintendent, there is 
no such mandate. The school has switched to the Scott-Foresman reading series model but they are 
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also continuing to focus on reading. Not until District 26 got involved with EMSTAC, however, did 
they tie their focus on reading directly to special education (both those students already identified and 
those at risk for being identified). They are also using this partnership with EMSTAC as a chance to 
bring in people with expertise to help them determine what needs to be improved and how to go 
about it. 

The LA’s primary goal this year is to bring awareness to disproportionality. As a school 
project, she hopes they will set an example for other schools. To help this process along, she hopes to 
install benchmarks. In addition, she also wants to link other facets of the business and education 
communities to the activities being undertaken at the school. She wants to ensure that resources are 
available and they have goals and objectives that are consistent with those of the 100 Black Men of 
America, Inc. She also wants to ensure their activities and progress are documented. 

The LA is working hard to help the school’s efforts to create and implement numerous 
activities that will help the school meet their goals around disproportionality and reading. Most of 
these activities focus on reading because poor reading skills hinder student learning across all 
disciplines. A graduate student is observing classes and analyzing the different approaches being 
used to teach reading. In addition, the graduate student is working with some of the special education 
teachers to help determine if specific accommodations should be implemented. Four students in a 
teacher training program in District 26 are having a lot of success working with four boys at the 
school who have been identified as having difficulties learning and succeeding by spending one-on- 
one time with them. The school is partnering with the city’s library system in a program, “Mother 
Read,” that teaches parents how to read effectively with their children. In addition, they have 
established an after-school tutorial for their “quartile two” (scored between 25 and 50 on their state 
standardized test last year) students that targets reading strategies and techniques to help them 
improve for this year’s testing. In undertaking and implementing all of these activities, the LA spends 
all of her time working with school administrators and teachers. 
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During the time that District 26 has been working with EMSTAC and the 100 Black Men of 
America, Inc., they have advanced through Havelock’s process of change. When they initially came 
on board with EMSTAC, they were in the “Relate” stage. Through their partnership and the support 
they have received from the Wimberly Initiative the district has been able to progress through the 
“Examine” and “Acquire” stages and is presently in the “Try” stage. 

1 

Important Factors 

The 100 Black Men of America, Inc. provides critical support for the school. They provide 
financial, time and personnel resources to help with the implementation of some of the initiatives to 
improve the outcomes of students. They fund the after school tutorial program as well as the graduate 
student who is observing and doing analysis of their teaching strategies. Beyond the 100 Black Men 
of America, Inc., the school is also drawing on the resources of EMSTAC, the neighboring 
university, and the city’s library system to help with the multiple programs they are running out of 
their school. 

When the school began working with the 100 Black Men of America, Inc., the district sent a 
representative from the district's office of special education to educate members of the 100 Black 
Men of America, Inc. on the issues the department was facing because most of the members of the 
100 Black Men of America, Inc. are not in the field of education. The department also trained some 
of the members to be “surrogate parents” to sit in on the M-team meetings (equivalent to lEP 
meetings). 

The LA has found that school and district administrators have been very open to the change 
associated with the activities she has been implementing through the assistance of the 100 Black Men 
of America, Inc. and EMSTAC. The LA has a few ideas of what the district could be doing to help 
further support her efforts in coming years: namely, to help the school to identify where its 
weaknesses are and to help foster more interaction between teacher training programs in the area and 
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teachers already in the field, and to get the business community more informed and more sensitive to 
issues of special education. 

The only barrier the LA was concerned about was teacher reaction. Since she herself is not a 
classroom teacher, she was concerned that teachers may not see her as being able to relate to what 
they go through as teachers on a day-to-day basis. The LA training, however, taught her techniques 
for engaging teachers and getting teacher buy-in. Right from the start, she engaged a special 
education teacher with whom she had a friendly relationship and who she knew if she could get his 
support, she would have enough credibility to at least get other teachers to listen. She included him in 
the organizing group to formulate the structure of the initiative, which helped to get the teacher 
excited about the project. His enthusiasm and support helped the LA to get support from other 
teachers. 

EMSTAC Support 

Due to the school’s involvement with the Wimberly Initiative, the LA has almost exclusive 
contact with the Chair of Educational Policies & Programs at 100 Black Men of America, Inc., one 
of the organizers of the Wimberly Initiative. She talks to him about once a week, mainly to touch 
base or if she has a problem or question. He has been a critical resource for both the LA and for the 
success of the programs being implemented. For example, when the school was establishing a parent 
room, he provided her with a lot of useful information and relevant trainings. The LA has found that 
the most helpful aspect of EMSTAC’s support in her activities is when they met and another LA 
involved with the initiative gave a presentation on how she tied her role into the initiative. That gave 
the LA a better understanding of her role and also the opportunity to ask questions and have a 
discussion, which was invaluable. 
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The LA has not had the opportunity to participate in EMSTAC’s listserv or on-line chat 
events. She has not had time to use the website to its capacity but she does feel that the information 
provided looks helpful if she could find the time to utilize it. 

The support that the 100 Black Men of America, Inc. and EMSTAC has provided has been 
very crucial to the school’s success with implementing multiple programs. “ Without them we would 
not have been able to come as far or be as successful.” For example, two of the three teachers who 
run the after-school tutorial initially would not even send home the permission slips. Once they got 
involved, however, they were very excited about the program because they came to understand the 
initiative and its importance. Since then, one of these two teachers has sought help for one of her 
students by getting him involved with the one-on-one help that the student teachers are providing. 

The LA has attempted to step back from taking the lead in some activities because she 
recognizes that this initiative is the 100 Black Men of America, Inc.’s. However, she is doing a lot 
since few of the members are part of the education field. The Chair of Educational Policies & 
Programs at 100 Black Men of America, Inc.’s attitude and actions have set the tone that the LA and 
others are striving to meet. He has inspired them not to quit and has shown her through his actions 
that nothing has to be a losing battle. She is driven to work hard because she never wants to fall short 
of how much the Chair does and how hard he works. The principal of the school has also inspired the 
LA because of the amount of time and effort she puts forth. 

One area of contention for the LA was the training itself. While she found the content to be 
very informative and helpful, she did not like having the training on-line. She felt that it was not 
interactive enough in the same way it would have been if she had been trained in a group setting 
where questions could be addressed face-to-face and discussions could take place. In setting up the 
Wimberly Initiative, there were multiple face-to-face meetings so having to do the EMSTAC training 
on-line did not seem to be consistent. 
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The LA and the school will continue their work surrounding disproportionality through the 
Wimberly Initiative and the 100 Black Men of America, Inc. Additionally, they continue to 
emphasize reading and increase activities and opportunities for parents. For example, the LA is 
working to continue to offer classes for parents and to set up computer training classes for parents. 
She usually holds four classes a year but because of her extensive involvement in activities 
surrounding the Wimberly Initiative this year, she was only able to offer one. 
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District 27 



District Background 

The District 27 County School District (Strategy III site) is a small rural district. The district 
encompasses three counties, each with their own political climate and culture, and two communities, 
including District 27. Because District 27 Municipal Schools encompass varying counties and 
communities, they serve a wide range of students - from well-established ranching families to very 
low-income families. The district includes seven schools: five elementary schools, one middle 
school, and one high school. Total enrollment in the school district is 4,360 students of which 11.4 
percent receive special education services, 34.6 percent are minority students and 3 1 .4 percent 
students qualify for free or reduced price lunch. Currently, students receiving special education 
services are almost exclusively served in separate classrooms, but the district is hoping to move to 
more inclusive environments soon. 

The district became involved with EMSTAC after one of our partners contacted the Director 
of Special Services in District 27 who expressed interest in participating. The Director of Special 
Services selected a special education teacher to be the Linking Agent (LA) and she went through the 
training. Upon completion she realized that being an LA was beyond what she could handle on top of 
her responsibilities as a classroom teacher. In response, the district transferred the LA role to a newly 
hired district administrator responsible for program development who reports to the Director of 
Special Services. The new LA brings to this role a wide range of experiences and enthusiasm. The 
2001-02 school year has been her first year with the District 27 schools, and her first in the LA role. 

District 27 Municipal Schools have had mixed experience with other improvement initiatives 
in the past. This was partly due to the lack of a qualified Director of Special Education for a period 
before the current Director of Special Services was hired three years ago. The current Director of 
Special Services is very driven and focused on implementing initiatives designed to improve district 
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capacity to accurately identify students who need special education services, monitor the 
achievement of these students, and improve their learning skills so they can exit from special 
education. A critical part of this process is building teacher capacity to implement research-based 
interventions and other practices that will help them effectively identify students with disabilities. 
This is hindered by the fact that teacher knowledge and experience in the district is uneven. As these 
forms of capacity are developed they will better enable the district to implement their priorities and 
initiatives. 

Many efforts have been made to formally adopt initiatives, but few have come to fruition. For 
example, this past year one school applied for a grant from the United Way to implement an 
alternative intervention program targeting students with behavioral problems. They did not receive 
the grant; however, the LA is still hopeful they can implement the plan on a smaller scale. 

Under the direction of the Director of Curriculum and Instruction and the Director of Student 
and Staff Accountability, the district has recently created a targeted curriculum. This curriculum 
follows a very specific scope and sequence of skill development for each child, inclusive of special 
needs children. So far, their focus has been primarily on the elementary grades, but it is expanding. 
Each child has a cumulative record specific to grade level, which identifies the skills that need to be 
mastered. This is the first full year of implementation but so far the feedback and results have been 
positive. 

One strength of the district is the teachers themselves. The Director of Special Services has 
worked to identify teachers who can help mentor younger ones and is hoping to link their lower 
incidence populations, visually impaired and hearing impaired, with itinerant teachers from the state 
School for the Blind and School for the Deaf In addition, each school site has a technical 
representative assisting school personnel with computer concerns. Their extensive knowledge and 
capabilities have been very helpful to the LA and others in overcoming barriers to getting the district 
online. 
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The district also sees their students as a resource. The high school is offering a class in a 
computer language, and students taking this class are contributing extra time wiring the entire high 
school. One school is offering its sixth graders a computer class that teaches them how to build 
computers. The district hopes this will save them money as students develop technology that can be 
used by others, and will enhance student computer savvy at a young age. 

The LA’s efforts have been focused on getting the district online and strengthening the 
knowledge and comfort level of teachers in using the resources available to them. Thus far she has 
not been utilizing external services of technical assistance beyond her partnership with EMSTAC. 
Last semester she spent most of her time building good communication skills with her special 
education staff and meeting with all her teachers and site administrators. This semester she is 
focusing most of her efforts on compiling data for a planning matrix and connecting teachers with 
what is available on the district’s network and through EMSTAC. 

Stage of Change 

Following the Havelock model of change. District 27 Municipal Schools is in the “Try” stage 
and is trying to work towards the “Extend” stage. They have particular goals and aspirations they are 
seeking to implement, and are focusing on building the basic infrastructure and capacity needed to 
put these goals in place 

District 27 Municipal Schools is currently working to improve their online capability by 
building a technological network within schools, between schools and to the outside world. They 
want their students to be not only aware of technology but also technologically savvy. This initiative 
started before the district began working with EMSTAC. They have been able to implement this 
initiative in a relatively short period of time. Three years ago, the district was not online at all. By the 
end of the 2001-2 school year, all schools except one elementary school will be online. This initiative 
began in response to issues such as the deficit in computer skills their students experience when they 
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enter college, the large number of special education students in the district, and the inundation of 
paperwork. Also, a year and a half ago the state created an additional competency for teacher 
licensure, technology, which has help spur the process on. 

The LA sees her main gpal this year as disseminating information. The way she describes it, 
she is helping her teachers “work smarter, not harder.” She is hooking up special education teachers 
in the district to resources through her interactions with EMSTAC. The LA passes on all the 
information and resources from LA-Exchange communications to the teachers she works with. Most 
helpful so far has been the resources on reading. Although she was already familiar with some of the 
information that EMSTAC has supplied, the information has been helpful for the teachers with whom 
she is working. She is also creating a website with links and reviews of multiple sites and resources 
the teachers can utilize. 

To assist the district in documenting the progress they are making in getting their classrooms 
and teachers online, and utilizing the online resources, the LA has created a detailed matrix that 
records the status of the technical connections, knowledge, and use by teachers for the entire district, 
K-12. 

The LA has a very hands-on approach to working with the teachers. She tries to go into each 
of the classes regularly and work with teachers to make sure they understand how to use the online 
resources the district is providing. Most critical is the district network, which includes the target 
curriculum and numerous resources for implementing it in their classrooms. 

Following the Director of Special Services’ belief that special education and general 
education is a partnership, the LA has been working with general educators to help them understand 
special needs children. 

Currently, the district is especially focused on the actual physical setup of getting staff online 
and training teachers on how to use the technology. As this is completed in the near future, they will 
move more towards implementing research-based strategies. The LA has worked with teachers to 
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find ideas and resources through EMSTAC but it is too early to tell how well they have been 
implemented in the classroom. 

Important Factors 

A number of factors are affecting the district’s progress in implementing the technological 
initiative in their schools. The administration and school staff are invested in getting all schools 
online and ensuring that all teachers are able to access and utilize the district network. The Associate 
Superintendent of Curriculum and Instruction has gotten behind this initiative and has helped in its 
implementation. The LA shared that of the 52 teachers she is working with and documenting in the 
matrix, only one is adamantly resistant, but this individual is team teaching with a computer literate 
teacher. Overall, this high level of shared goals and commitment, with support from district leaders, 
is helping the district move forward with their plans. In a similar way, some members of this small 
rural community are concerned about being electronically connected with unknown people and 
sources outside of their own area, and worry about the negative influence this may have on their 
community. District leaders are sensitive to these concerns and move with appropriate caution where 
needed, but are also working to help the community better understand how these connections to the 
bigger world will provide new opportunities for economic growth. 

EMSTAC Support 

The LA said that she has found EMSTAC helpful as a resource for both herself and the 
teachers she works with. The LA has found identifying research-based materials off the web site and 
from the LA Exchange to be especially helpful, and has sent them on to her teachers. She has also 
used the EMSTAC web site with teachers to find answers to their questions. She did note that the 
resources she has gotten have occasionally been too fundamental for her, though beneficial to her 
teachers. She has not taken part in any of the online chat events, due to the district’s firewall, though 
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she has printed out all the transcripts for both herself and her teachers, who find them very helpful. 
Because the district is still in the process of establishing their online capabilities, their interaction 
with EMSTAC has not moved forward as much as they would like. 

Both the district and the LA seem to be very positive about their current and future 
relationship with EMSTAC. They view EMSTAC to be a valuable resource they hope to tap into 
more extensively in the future. As the Director of Special Services described, the difficulties that 
District 27 Municipal Schools have faced in laying the infrastructure to establish a network in the 
district demonstrates why “. . .we are really just at our infancy with really wanting to grow with what 
EMSTAC can do for us, as a resource for teachers.” 
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EXHIBIT A-26: 
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Delivery Model, 2001-2002 
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District 28 



District Background 

A telephone interview was conducted with the Linking Agent (LA) of School District 12 at 
the beginning of May 2002. This district has approximately 13,500 students - 15 percent of the 
student body receives special education services, 99.2 percent are minority students, and 91 percent 
are eligible for free or reduced price lunch. The district became involved with EMSTAC after the LA 
attended a conference presentation by EMSTAC staff The LA currently works as a guidance 
counselor at an elementary school in the district and has been in the field of guidance counseling for 
the past ten years. Although the LA works primarily with general education students, he also 
collaborates with the guidance counselor who works with the students receiving special education 
services to ensure that all student needs are appropriately addressed. 

Over the past few years, the District has focused on initiatives that are 1) more consistent 
with the goals and 2) in compliance with the Individuals with Disabilities Education Act (IDEA) 
Amendments of 1997. For example, the District has been working to include more of the students 
receiving special education services in the general education classroom. The District has been 
working with two pilot schools to “try out” implementation of the practice. Within these schools, all 
children receiving special education services are included in the general education classroom for the 
entire school day. In the elementary school where the LA is employed, a proposal for additional 
funds was written to hire a special educator to help support an inclusive school environment. This 
teacher would co-teach in a general education classroom for 60 percent of the school day and would 
facilitate staff development activities and community and family outreach programs for the 
remaining 40 percent of the day. The school is still waiting to find out if they have been awarded the 
grant. Finally, the school and the District are conducting more regular meetings among general and 
special education personnel about students with disabilities and Individual Education Programs 
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(lEPs) and have implemented programs to improve the planning process. The goal of these programs 
is to improve the educational planning of students’ lEPs. 

Stage of Change 

Beginning a few years ago, the LA observed an increase in the number of students referred 
for pushing, fighting, or other behavioral problems. In response to these problems, the LA felt a need 
to research and develop a program appropriate for his school to address the conflicts that the students 
were experiencing at school. The LA identified two commercial conflict resolution curricula, one 
focused on grades K-4 and the other on grades 4-8, and used these curricula as a foundation for 
developing an appropriate program for his school. 

The program he developed is entitled “Freeze Frame” and the LA has been implementing the 
program in his school for about four years. The primary goal of the program is to give students the 
necessary tools to handle conflict; it includes worksheets and posters that show different techniques 
for resolving conflict. The program teaches students different ways to handle conflict by encouraging 
the use of “I” statements and role-playing. The LA began his program by building a foundation of 
knowledge about conflict resolution through using the books from the two commercial curricula and 
then adding to the curricula by incorporating new ideas, such as multiple intelligence theory. 

The “Freeze Frame” program engages students in role-playing activities that involve 
situational conflicts that may arise throughout the school day. As students role-play the provided 
scenario, the audience is expected to tell the role-play students to “freeze” when inappropriate 
behavior is displayed. The audience is then asked to offer a solution to the problem that reduces the 
conflict and “fixes” the problem. 

The program has been very successful and has been featured on a local television station as 
well as the district newspaper. Several people within the District have come to see the program in the 
school and the LA has been invited to other schools to conduct staff development sessions using the 
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techniques of the program with other counselors and teachers. The LA prepares the monthly reports 
that include the number of incidences of conflict that are reported. The number of incidences has 
drastically decreased since the inception of the program from two to three per day to two per month. 

According to the Havelock change model, this initiative appears to be entering the “Extend” 
phase. The LA has been implementing, or “trying” the program for the past four years, incorporating 
adaptations when necessary. Although the LA has done some initial work to “spread the word” about 
the initiative, more staff development and outreach work are needed to foster the use of this program 
throughout the District. The LA is collecting data on number of referrals and outcomes of the 
intervention to evaluate the overall effectiveness of the program. The LA hopes to continue to work 
on promoting the successes of the program and train other educational personnel in the District in the 
techniques of “Freeze Frame.” 

Important Factors 

One of the most important factors in the implementation of this program was the support that 
the LA received from the teachers and administration. The LA believes that this support comes from 
the willingness and openness of both the District and the school to new initiatives and to positive 
change. The teachers and principal of his school provided support and gave him class time to 
demonstrate and implement the intervention. In addition, the district administration, more 
specifically. Student Support Services, arranged for him to conduct staff development events in the 
area of conflict resolution. He is getting additional support by receiving new training on conflict 
resolution through the district. 

The LA felt that one of the main barriers to implementing this initiative has been the heavy 
emphasis placed upon testing in the school. Teachers are focused on teaching to the tests, and 
therefore do not wish to take the time to incorporate lessons for conflict resolution. Like many LAs, 
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he would like to be provided with more release time to implement his program and increased 
opportunities to share the information with other colleagues in the school district. 

EMSTAC Supports 

The LA was appreciative of the support that he received from EMSTAC this past year. He 
has found that the most beneficial aspects of the support he received from EMSTAC have been the 
materials and resources sent to him by his TA Liaison, in addition to the materials he has accessed 
on-line. He receives e-mail from his TA Liaison on a regular basis, about two to three times per 
week, and has found the listserv for LAs helpful. Finally, he found the training helpful for reinforcing 
some ideas that he had learned previously and providing him with new ideas for improving his ability 
to act as an LA in his school district. 

The LA is concerned about the future of “Freeze Frame.” He has applied for an 
administrative role for the next school year and does not know how much time he will be able to 
devote to this specific initiative once in the new position. He is currently trying to ensure that 
information about the program is disseminated throughout the District and hopes that other personnel 
in the District will be able to carry out implementation of the program in the future. If the LA does 
accept an administrative position, his LA training and skills learned will be useful in helping him to 
identify new areas of concern and conduct new needs assessments to determine what initiatives 
might be most appropriate for continuing to foster positive change there. 
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EXHIBIT A-27: 

District 28 Technical Assistance 
Delivery Model, 2001-2002 
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District 29 



District Background 

The District 29 School District is a Strategy III district. It is one of 40 school districts in the 
state, with a population of approximately 3,923 students encompassing four elementary schools, two 
middle schools, and a high school. The Linking Agent (LA) reported that in 1990 there were 129 
students pre-kindergarten through age 22 who received special education and related services. That 
number has increased to 450 students today. The district has 20 special education teachers, including 
three speech therapists, and slightly over 200 general education teachers. Of the total student 
population in District 29, 10.2 percent receive special education and related services. Almost nine 
percent of the school district’s student population receives free or reduced price lunch and roughly 88 
percent are of racial and ethnic minority families. 

The LA serves many roles in his school district. In addition to being the Director of Special 
Education for the District 29 School District, he serves as the Personnel Director and is in charge of 
the English as a Second Language/English Language Learners (ESL/ELL) program, as well as the 
At-Risk program. He said wearing “several hats” is a common arrangement in small districts in the 
state, and one that affects his ability to focus on special education and use the support provided by 
EMSTAC. His time is spread across several responsibilities. 

The LA first learned of EMSTAC through a phone call from a senior EMSTAC staff 
member. After a couple of phone conversations with the LA, the EMSTAC representative attended a 
state special education directors’ meeting, where both met face-to-face. During that time, the LA 
expressed an interest in participating with EMSTAC. The concept of having cutting edge information 
at his finger tips attracted him to EMSTAC. Also, he wants to use EMSTAC as a vehicle for 
exposing special education teachers to new ideas. 
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Over the past year the district has launched into a focus on building the capacity of teachers 
to engage in differentiated instruction. The LA sees this as the key to becoming one system where 
instruction occurs in inclusive settings for all children, and only limited opportunities for self- 
contained classes. He also sees differentiated instruction as a safety net for children in the early 
elementary school grades. During this past year training was provided to general and special 
education teachers on differentiated instruction. 

Reading has been a major initiative over the past several years. In various ways the district 
has attempted to address the need for reading improvements for students in kindergarten through 
sixth grade, especially for students who are English language learners (ELLs). Among the initiatives 
used in the school district to improve reading are the following: 

• SRA programs, which have been used in the district for many years (the LA commented 
that these programs are not being used “in the true sense.”) 

• One teacher is in her second year of piloting Read Well, a Direct Instruction (DI) 
structured, phonetically based program. 

• Early Steps, which will be implemented Fall, 2001 in all K-3 classrooms. The LA 
described this modified Reading Recovery program as less staff intensive than other 
programs and includes better accountability structures than other programs. Because of 
his concern with a lag in reading development, the LA wanted to focus efforts on K-2 
students to prevent reading problems and avoid remediation as much as possible. 
According to the LA, the district’s Curriculum Director and four reading specialists 
reviewed and selected this program. The LA commented that “the weak link is that no 
special education staff were involved” in the selection of this intervention. He said that 
his hope for this program is for “continual assessment and re-teaching” of reading. He 
added that teacher training for this program has met with resistance, especially from those 
teachers who align themselves with the “whole language approach.” 

• Boystown Reading Program, which began two years ago in the 7th and 8th grades (and in 
high school). Students who may best benefit from this program are selected to participate. 
The LA said that although the district’s middle schools include grades 6-8, the Boystown 
program addresses only 7th and 8th grade students. He expressed concern for addressing 
students with similar needs in the 6* grade. 

Both the state and District 29 have had an increase in their ELL population, which the LA 
describes as exacerbating the need for reading improvement. About six to seven years ago, there 
were approximately 20 ELL students in the district — mostly of Hispanic descent. Currently, the 
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district has over 250 ELLs. This ELL population growth has increased from approximately one to 
two percent to 25-30 percent in one school alone. 

Next year the district plans to initiate a program focused around student career paths. In this 
program students will select a career path associated with a given set of educational experiences and 
goals. Details have not been fully developed, but the LA believes it will have implications for the 
academic curriculum. 

The LA said lack of time, financial resources, and consistent school level support are barriers 
to moving forward with these initiatives. Time is critical in a number of ways. By wearing several 
hats, the LA said he does not have sufficient time to follow through and work with school 
administrators and teachers on a sustained agenda. Time is also a problem for teachers who face lots 
of pressures from different sectors. It’s difficult to pull teachers out of their classrooms for training, 
or for them to get involved in planning with other teachers or resource specialists, and collaborative 
activities with other groups. Another problem is the lack of financial resources necessary to pay 
teachers sufficient salaries that would enable them to live in the district, which is a high income 
community. This affects their ability to recruit and maintain a high quality staff He also said school 
administrators and teachers do not always have clear consensus about their needs or support for the 
goals of district initiatives. 

In a previous interview the LA said state funding for these programs is very limited, most of 
it targeted for hiring reading specialists and class-size reduction. The LA said the school district 
obtained a state grant for systems change implementation. Through the grant, the school district 
selected and trained teams of general and special educators to address school needs involving the 
inclusion of students with disabilities in general education curricula. The teams surveyed schools to 
determine how well inclusion is working. The LA is pleased overall with how inclusion is being 
handled in the elementary and middle schools, saying that the district is “fine-tuning” the details of 
the structure so that it runs more smoothly and can better impact on the high school level. 
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Other forms of external support that the school district has used include those provided 
through the state Learning Center, which provides mostly staff-development services and assistance 
with individualized case consultation, and occasional training. The LA said that although he utilizes 
the Learning Center from time to time, he does not ordinarily do so. 

Overall, his impression of the services provided by the state is negative. He expressed 
concern that the state director of special education does not have a background in special education. 
He believes a system of TA involving national-state-local partnerships would have potential for 
widely and effectively disseminating information if it is handled the right way with qualified 
professionals who have a background in the field and are focused on addressing local needs. His 
experience is that too often state level TA is inadequate and gets weighed down by excessive 
bureaucracy. 

Stage of Change 

In reflecting on these issues, the LA said the initiatives they are implementing are not 
occurring, though he sees promise for EMSTAC to meet the district’s need for moving research to 
practice in their classrooms. The LA considers quick access to information to be the most important 
aspect of EMSTAC involvement. 

Identified needs in the school district are based on what school faculty report to him through 
e-mail and scheduled periodic meetings. Each day he receives about two to three e-mail messages 
from faculty with questions. Additionally, every other month he meets with teachers at the school 
sites to discuss staff development needs. He said that although general and special education staff at 
the elementary schools have expressed a need for reading improvement, he wants them to determine 
what specific aspect of this area needs to be targeted. Despite these efforts, he is disappointed with 
the process used to identify needs. He believes a more comprehensive, systematic process is needed 
to identify faculty needs. 
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With regard to reading instruction in general, the LA said that the school district has had 
problems identifying reading difficulties, finding materials and approaches to enhance reading 
improvement, ensuring that materials are individualized to meet the needs of each child, and overall 
parental fhistration. 

EMSTAC Support 

Thus far the LA has not had an opportunity to begin providing EMSTAC technical assistance 
in his school district, and he has accessed EMSTAC support on a very limited basis. He suggested 
EMSTAC’s services could be improved by showcasing studies of districts EMSTAC has worked 
with. In a previous interview he recommended publishing and publicizing examples of how 
EMSTAC has interacted and helped school districts so that other LAs and school districts understand 
the possibilities and observe the reality of bringing about change in schools. In this interview he 
made a similar suggestion, citing the value in learning about initiatives in other districts. He 
suggested this could be packaged in a monthly electronic newsletter that could serve as a prompt. He 
said he needs a prompt that alerts him to what is available. Because he wears a number of hats and 
feels pressed for time he is unlikely to take advantage of a TA system that depends on him taking the 
initiative. He also said that when he initially began working as an LA he was more self-assertive and 
researched a couple of issues on the EMSTAC website, but found the information did not cover the 
topic with sufficient depth to make it useful. He said one way to build greater depth would be to 
establish layers of linkages to other sources. Through these multiple sources the EMSTAC website 
could provide in-depth information on several topics. He believes that having quick access to critical 
information is essential to mobilizing change. 

In a previous interview the LA said that he found the training to be “well organized and easy 
to use.” He further commented that the training “contained useful information.” He said he 
appreciated how well organized the training was presented and said that he had no difficulty 
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navigating, reading, and accessing the training site and materials. He said that the interactivity of the 
training “added to the content.” Although he said that he found the training to be lengthy at first, and 
that he had difficulty scheduling time to complete the training, he said, “I’m not sure how you can 
change that.” He considered the process of change introduced in the training to be “a great 
refresher,” and said that the training clearly defined his role and mission as an LA. 
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EXHIBIT A-28: 

District 29 Technical Assistance 
Delivery Model, 2001-2002 
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District 30 



District Background 

District 30 Independent School District is a large, urban district that serves 57,621 students in 
93 schools. Of these students, 13.7 percent receive special education services, 95.7 percent are of 
racially and ethnically minority families. According to the District 30 Independent School District 
website, 93.3 percent of the students in the district are considered economically disadvantaged. 
EMSTAC is working with one middle school in the district through the Wimberly Initiative with the 
100 Black Men of America, Inc. The school is located in a neighborhood of single-family homes and 
also serves four housing projects. As of fall 2001 it is 44 percent African American, 52 percent 
Hispanic, and four percent Caucasian. 

District 30 became involved with EMSTAC through the work the district was already doing 
with the 100 Black Men of America, Inc. In the spring of 2000, the Special Education Director of the 
district met with representatives from the 100 Black Men of America, Inc. They decided to work 
together and pilot the initiative at the middle school in fall 2000. The partnership between the 100 
Black Men of America, Inc.’s Wimberly Initiative and EMSTAC began spring 2001. The goal of the 
Wimberly Initiative is to provide supports for students at risk of being evaluated for special education 
placement to enable them to remain in the general education classroom. 

The school chosen to participate in District 30 is unique within the initiative, however, in that 
they work with students who have already been identified for special education and it is the only 
middle school. Therefore, the goal at this site is to provide appropriate supports for students in 
special education to improve outcomes and enable them to return to the mainstream. 

The Linking Agent (LA) got involved with the initiative in the early phases of the initiative. 
She met her TA Liaison and participated in the online training between summer and fall 2001 . By the 
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fall of 2001, the school had been disestablished to improve conditions at the school; therefore the 
initiative was implemented with a new principal and new faculty. 

The administrator and staff are also working on other initiatives related to special education. 
This is the first year that they are implementing a full inclusion model. There is one resource class for 
each subject area (e.g., reading, math) for students functioning at the second grade level and below. 
Beyond that, all students are placed in inclusive environments. The school and district supported this 
initiative by providing additional teacher assistants to help with the transition. The school continues 
to have two self-contained behavior improvement units, one self-contained autistic unit, and a unit 
for children with severe disabilities. The LA is unsure about what, if any, initiatives were enacted 
prior to the school’s reorganization. 

Stage of Change 

The LA’s main goal this year has been to provide reading interventions so students can be 
removed from special education or be tested with the general education population. The school began 
working with EMSTAC because they saw it as an opportunity to have more access to in-depth 
research and more practical ideas that they could utilize as they work on designing their plans. The 
LA acknowledged that they have not been able to utilize EMSTAC’s resources to the degree that 
they had envisioned because of factors outside of EMSTAC that have prevented them to do so. For 
example, there has been a new principal at the school in seven of the last eight years. With the latest 
change in the school’s administration this year, came a change in the focus of the Saturday Academy. 
The 100 Black Men of America, Inc. provide valuable resources to promote success of the initiative. 
In addition to the financial contribution to pay for supplies and staff for the Saturday Academy, their 
members work with the students each week. Also, the 100 Black Men of America, Inc. provide 
breakfast and lunch for the students each week as well as rewards for good attendance and grades. 
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In addition to EMSTAC and the 100 Black Men of America, Inc., the school also works with 
several mentor groups in the area. Some of these groups focus their work with the school solely on 
children in special education. District support staff, teacher specialists, also work on-site at times. 
While both of these are helpful resources, the LA notes that the district could help even further by 
providing additional teacher assistants to assist with the transition of students into inclusive settings 
and training general education teachers to provide modifications for the special education population. 

The progress of the students involved in the initiative is collected, recorded and tracked every 
nine weeks. The LA notes that the results from the first year of the academy seem to indicate positive 
outcomes. One reason for this, she suggests, is that that there were more students last year and so 
they could have appeared to progress at a faster rate. Even so, both years have had a noted impact. 
Additionally, there are more students in special education who are taking the general education state 
assessment. 

The LA gives much of the credit for student growth to the skills and knowledge of the 
teaching staff This year the school faced several staff changes including some key team members. 
Two of them taught the Saturday Academy last year and were difficult to replace. Another challenge 
has been the change in administration. 

During the time that District 30 has been working with EMSTAC and the 100 Black Men of 
America, Inc., they have advanced through Havelock’s process of change. When they initially came 
on board, they were in the “Care” stage. One need identified was addressing the overrepresentation 
of minority students in the special education program. Through their partnership and the support they 
have received from the Wimberly Initiative, the district has been able to progress through the 
“Relate,” “Examine,” “Acquire,” and “Try” stages and is presently back in the “Examine” stage. In 
the “Relate” stage, they established contacts and built relationships within the system and external to 
the system. In the “Examine” stage, District 30, through its participation in the Wimberly Initiative, 
analyzed disproportionality in their schools more closely and formulated a plan of action. In the 
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“Acquire” stage, they accessed numerous resources, including those from EMSTAC. As they 
implemented interventions to address the problem, they were in the “Try” stage. The 
disestablishment and reorganization of the school influenced the change process. They are now once 
again in the “Examine” stage, where they are revisiting the successes and challenges that they have 
faced in order to improve the program. 

Important Factors 

The disestablishment of the school brought in a new principal and replaced all staff in fall 
2001. As noted above, the current principal is the seventh new principal in eight years at the middle 
school. As a result, the priorities shifted to those that are more consistent with the new administrator's 
plan. The new principal has influenced and changed the focus of the Saturday Academy. The LA 
indicated that she lost some of her biggest advocates and best teachers. In addition to the 
reorganization, the school initiated implementation of full inclusion this year. Due to the strength of 
the 100 Black Men of America, Inc.’s support, both financially and otherwise, the initiative has been 
implemented and has shown a high degree of success. 

EMSTAC Support 

The LA has a good relationship with her TA Liaison. Their contact was more extensive 
during the first semester of the year, about once every two weeks. Due to some family circumstances, 
however, the LA was away for part of the second semester and, currently, she and her TA Liaison 
have not reestablished communication to the same level it was previously. Currently, their 
communication is comprised of e-mails on a less frequent basis. They have a variety of types of 
interaction, with the most frequent one being touching base. The TA Liaison also made one visit to 
the site in fall 2001 . The LA meets with the President of the District 30 chapter of 100 Black Men of 
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America, Inc. at least once a month and has contact with the Chair of Educational Policies & 
Programs at 100 Black Men of America, Inc. as needed. 

The LA has utilized many of EMSTAC’s resources. She finds that the training and listerv 
have been the most beneficial. She notes that she has received “a lot of really great things” from her 
participation in the listerv. Unfortunately due to scheduling conflicts, she had not been able to 
participate in any online chat events. She does, however, use both the public and private side of the 
website and she finds the private side to be more beneficial. In addition, she has requested written 
products from EMSTAC which have been very useful. She noted that she found the TA Liaison’s 
motivation to move the process along to be very helpful. 

Through the Wimberly Initiative, this school has been able to focus some of its efforts to 
support their special education population by working to enable more of these students to exit special 
education and reenter the general education program. The support the school has received through 
the initiative from the 100 Black Men of America, Inc. and EMSTAC has been critical to its success. 
While they have faced obstacles due to changes in personnel and administration at the site, they have 
continued to persist and improve children’s lives. 
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EXHIBIT A-29: 

District 30 Technical Assistance 
Delivery Model, 2001-2002 




Exhibit Key 
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District 31 



District Background 

District 31 is located across the border of two states. Within its area, the EMSTAC Linking 
Agent (LA) is assigned to eight schools, two of which have attempted to use EMSTAC resources and 
supports. At these two schools, the LA has been working to implement and support school-wide 
positive behavior interventions. 

The LA reported that she learned about and became interested in EMSTAC due to a number 
of presentations she attended at national conferences in special education. She indicated that she felt 
that the resources provided by EMSTAC could be useful in helping to supplement information being 
obtained from other sources. In particular, much of the LA’s training and source materials have come 
from the OSEP sponsored Positive Behavior Intervention and Support Center. 

In May of 2002, EMSTAC visited the LA at one of these two schools. This particular campus 
includes a dormitory that houses approximately 30 percent of the student population. Students living 
in the dormitory range in grades from first through sixth. In addition to the current initiative, the LA 
reported that the school has a history of implementing programs intended to facilitate the academic 
and social skills of the students. A number of these programs have been successful but many have 
been difficult to sustain due to one or two individuals being primarily responsible for their 
implementation. Funding was also limited. 

Stage of Change 

The school is currently addressing student behavior through the use of school-wide strategies 
to decrease office referrals. This need was primarily chosen due to a general consensus within the 
building that behavior disruptions were at a problematic level and due to the recognition that there 
was little consistency in how disruptions were being addressed (or prevented). 
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During the current school year, the LA has worked with the principal, other support 
personnel and teachers to implement some specific support strategies. In addition to conducting 
training in positive behavior management and working to develop school-wide behavior policies, the 
LA has worked to collect and analyze data on the behavioral patterns on the campus. This 
information has been used to develop a set of school-wide rules and some common contingencies 
regarding positive and undesirable behavior. Finally, the LA has facilitated productive discussions 
between the school and dormitory staff that previously had not occurred. This has been useful in 
beginning to increase collaboration and understanding about the unique circumstances and 
commonalities across the two settings. 

As a result of the LA’s efforts, teachers and staff have begun using some effective behavioral 
support strategies. Similar efforts have reportedly occurred in the other school in which the LA has 
been addressing this need. At this point, the district appears to be in the “Try” stage of the Havelock 
cycle. 

Important Factors 

A number of factors appear responsible for the District’s success in implementing research- 
based interventions that address school-wide behavior. First, the LA reported that she has received a 
great deal of support firom her supervisor, the Director of Special Education, who has allowed her to 
attend professional development conferences and actively encouraged her to engage in system-level 
(school-wide) consultative activities. It should be noted that as a Behavioral Specialist, the LA’s 
primary responsibility is to provide consultation and intervention for individual teachers and 
students. 

A second critical factor appears to be the high level of collaboration with other professionals 
in the school building. The LA has worked in conjunction with a core group of other individuals to 
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ensure the development and implementation of the school-wide behavioral support strategies. In 
addition, the LA has actively sought input from teachers and staff members to ensure total buy-in. 

Finally, there are a number of highly skilled and dedicated professionals working on one 
campus; this has created an opportunity to establish local expertise that is often needed, particularly 
in the early stages of an initiative. In particular, the ability of a number of the staff to understand and 
articulate the need for and the reason behind a school-wide approach to supporting student behavior 
has been important in insuring that staff embrace and feel supported in implementing new strategies. 

EMSTAC Support 

The LA has accessed EMSTAC supports. Throughout the past school year, the LA and TA 
Liaison exchanged ideas via e-mail on approximately six occasions. These exchanges centered on 
strategies for reducing playground disruptions and generally increasing structure in other 
unsupervised areas. 

The LA reported that she has accessed and downloaded a number of web-based resources 
during the year. She has found the articles on school-wide discipline to be most useful but has 
accessed other resources listed throughout the website. The LA also indicated that she does read the 
e-mails posted to the list-serv but rarely has felt the need to respond to the posts. She has, however, 
sent a number of posts to the EMSTAC bulletin boards. Finally, due to difficulty accessing the 
Internet during the school day, the LA has not participated in chat events. She reported that she was 
interested in this medium but that it is difficult for her to find time during the day to use the 
computer. 

In general, the LA reported that EMSTAC supports have been useful in providing her with 
additional information and ideas. She reported that EMSTAC could be more beneficial to school 
districts if some personal contact were made throughout the year. Specifically, she felt that one face- 
to-face visit would be useful to demonstrate EMSTAC ’s commitment to a particular district. In 
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addition, she believes that this would allow EMSTAC to better understand local norms, values, and 
the school organizational climate. 
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EXHIBIT A-30: 

District 31 Technical Assistance 
Delivery Model, 2001-2002 
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District 32 



District Background 

On April 30, 2002 two EMSTAC staff members conducted a telephone interview with the 
Linking Agent (LA) for the District 32 School District. District 32 is a small district, with a total 
enrollment of 869 students. Of these students, 15.9 percent receive special education services and 
23.4 percent are of a racial minority. 

The LA became interested in working with EMSTAC after meeting an EMSTAC Affiliate 
from Parents Inc. at the Pathways Conference in the spring of 2002. She learned more about 
EMSTAC while attending a session about the collaboration of EMSTAC with Parents Inc. at the 
conference. The LA has taught in the District 32 school district for a total of eight years, and she has 
been a middle school resource teacher for the past four years. 

The district is currently planning to implement the Quality Schools Initiative, and they are in 
the process of selecting standards and measurements for each level. District 32 is using the Quality 
Schools Model that another School District successfully implemented, but they plan to tailor it to the 
needs of District 32. The school has recently held meetings to solicit parent and community input. 
They have met resistance from a small group of parents who feel that the District 32 should not base 
a program from the other district’s Model because the needs of the school districts are very different. 

District 32 is currently receiving support from the Quality Schools Institute for this initiative. 
They will be doing a Quality Schools Symposium with teachers as a two-week in-service before 
school commences next fall. In the past, the University of Iowa has also provided technical assistance 
to District 32 in regards to IDEA requirements. 

Stage of Change 
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The district is currently in the process of identifying curricular-based means of student 
assessment to monitor progress and to assist them in placing students in appropriate grade levels. As 
part of the Quality Schools initiative, these assessments would help the school to place students in 
appropriate levels based on demonstrated skills and achievement rather than using the grade 
structure. The LA stated that the teachers believe that “we are leaving students behind and socially 
promoting students to the next grade when they do not have the skills to move to the next grade.” In 
addition. District 32 is also looking into new reading programs as well as standards in reading. 

The LA is part of a team of teachers that is trying to implement the Quality Schools Initiative. 
She plans many of the meetings and gathers research and background information about standards 
for the group. Recently, the EMSTAC TA Liaison and the LA coordinated a conference call with a 
group of teachers, the superintendent of District 32, and an expert on reading and assessment from 
the American Institutes for Research (AIR). Prior to the meeting, the LA faxed the TA Liaison the 
standards that they are considering implementing. During the call, the participants discussed the 
standards and it was suggested that the group may want to consider developing more comprehensive 
reading standards that include all the components of reading such as fluency, phonemic awareness, 
comprehension, etc. Following the call, the LA and the TA Liaison have been in communication to 
operationalize the definitions of the components of a reading program and how these definitions can 
translate into standards for the schools. 

District 32 is currently in the “Acquire” stage of the Havelock change process. A group of 
teachers with the same concerns had been working together to examine standards and they are in the 
process of acquiring resources to develop standards. They are currently assessing the standards of 
other schools and reaching out to EMSTAC for further information about standards. District 32 plans 
to move into the “Try” stage of the Havelock model this fall when they begin to implement the new 
standards within the Quality Schools Initiative. While the LA and the school intend to move forward 
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in implementing the Quality Schools Initiative, they are also continuing to examine the problem and 
to acquire resources so that they are making informed decisions. 

Important Factors 

The buy-in from the teachers and community support have been important factors 
contributing to the progress of the district in preparing to implement the Quality Schools Initiative. 
The LA and the teacher, who have formed the work group to examine the standards, have been the 
impetus and leaders in identifying this need, acquiring resources, and moving the process along. The 
LA reports that within the school, teachers are already tailoring the curriculum to individual student 
needs. Additionally, at an in-service this year, 70 out of 80 teachers voted that they would like to 
implement the Quality Schools Initiative. The teachers’ support of the program has been extremely 
important in gaining administrative support and corroborating the need for the initiative. 

Community support and the buy-in from stakeholders also seem to be important for the future 
success of the program. At the first meeting to discuss the Quality Schools Initiative, a small group of 
parents expressed strong disapproval of the program. Despite the negative responses, the school held 
a second meeting in which they made a presentation to further inform parents, and they distributed a 
survey to solicit feedback from parents. Although the LA reported the Quality Schools Initiative has 
been meet with resistance by a small group of parents, the school has persevered in trying to inform 
the community and acquire stakeholder feedback. 

EMSTAC Support 

The relationship between District 32 and EMSTAC has come at an important time in which 
District 32 is seeking resources and can benefit from knowledge of the change process as they 
embark to implement a big change in the school. Recently, EMSTAC has been providing District 32 
with research-based resources to expand their knowledge base of reading programs, components of 



ERIC 



A-181 



352 



American Institutes for Research 



reading, and curriculum-based assessment. The LA also stated that, “EMSTAC’s support and 
training has helped prepare me for change and has also enabled me to help others with the changes 
that our school is going through.” The LA mentioned that she has not had a chance to go through all 
of the products on the EMSTAC web site. However, since the time of the interview, the TA Liaison 
has sent EMSTAC’s Selecting a Reading Intervention and the report. Preventing Reading Difficulties 
in Young Children by the National Research Council, via e-mail and hard mail to the LA. The LA 
has also reported that although she has not been able to get on the chat events, she has read through 
several chat transcripts that have been very informative. 

In conclusion. District 32 is examining resources to develop appropriate standards that 
will be a part of the Quality Schools Initiative. EMSTAC has been able to provide District 32 
with some research-based materials and support in the development of reading standards. The 
TA Liaison and the LA are continuing to coordinate next steps for the district to take to ensure 
that the reading standards are more comprehensive. 
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EXHIBIT A-31: 

District 32 Technical Assistance 
Delivery Model, 2001-2002 
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EMSTAC EVALUATION LINKING AGENT INTERVIEW 

FORM I SPRING 2002 



This protocol should be used to guide your interview with the Linking Agent. It is our intent 
that these questions are asked to each Linking Agent with whom EMSTAC is working. There may be 
times when certain questions are inappropriate, irrelevant, or redundant. In such cases these questions 
may be skipped. 



SAY : As you are likely aware, we are in the process of attempting to determine the most effective 
ways of providing technical assistance to school districts. Because your role as Linking 
Agent is seen as a critical element within the EMSTAC model, we wanted to spend about 30 
or 40 minutes getting your impressions about the Linking Agent role, your school district, 
and the type of support that EMSTAC has provided to you. 

I . Could you tell me a little about your background in this field? 



SAY : Now, we ’d like to know more about the different activities that Linking Agents, like yourself, 
are engaging in. We are also very interested in learning what kinds of support you receive 
for these activities, and how we at EMSTAC can better support your efforts. 

2. As a Linking Agent, what do you see as being your primary goal or goals this year? 



3. Please describe the program or initiative you are implementing. 



4. Was this initiative chosen before working with EMSTAC? (STATUS) 
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5. Where did the motivation come from for initiating this program? (District, school, mandate vs. 
voluntary) (LOCUS, MOTIVE; 3, 4, 5) 



6. Do you know why your (district or school) chose to work with EMSTAC? (REASON, 
MOTIVE) 



7. How effective have the interventions been in meeting their intended goals? How do you know? 
(Probe for evidence) (HAVELOCK; 13) 



8. What have been some barriers that you have faced in attempting to provide technical assistance 
to your school/district? 

(Probes) 

• Resistance to change? 

• Building support and consensus? 

• Identifying the TA topic 

• Prioritizing needs? 

• Conducting a needs assessment? 

• Identifying researcher(s)? 

• Logistics? 

SAY : OK, now let 's talk a little about your school/district.. . 



9. Is your district/school currently and actively implementing initiatives or reforms to improve 
outcomes for children in special education? (CURRENT) 
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10. In the past five years, has your district/school adopted initiatives or reforms to improve outcomes 
for children in special education (other than the current ones mentioned in Question 9)? (PAST) 



1 1. In terms of (identified need) what internal and external non-financial resources do you have 
available and are those resources helpful? (Probe for both internal and external resources). 
(COLLAB, EXTERNAL, RESOURCE) 



12. Specifically, what do the school and district do to support you? (BLDGLEAD, DISTLEAD, 
COLLAB; 8) 



13. (If not already answered) Did you have to secure outside sources of funding (outside of the 
district budget allocations) in order to start the program? How did you go about obtaining these 
funds? (EXTERNAL, RESOURCE) 



14. (If not already answered) Do you have external sources of technical assistance besides EMSTACto 
support this initiative? Please describe. (RESOURCE, EXTERNAL) 



15. Could you estimate how much time have you spent working with district administrators, school 
administrators, and teachers this year on this project? (Probe for % of time) 

Percent of time 



District administrators (DISTLEAD, COLLAB) 
School administrators (BLDGLEAD, COLLAB) 
Teachers (COLLAB) 
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16. On the scale below, please rate the extent to which individuals at the district and school level are 
open and willing to using external technical assistance related to providing services for children 
with special needs: 

Not Open to Change Very Open to Change 

1 2 3 4 5 



Rating 

District personnel (DISTOPEN) 

School personnel (SCHOPEN) 



17. What could your school district do to support you as a Linking Agent in future years? 
(COLLAB) 



18. Would you like to continue being a Linking Agent? Why or why not? 



SAY: Now we would like to find out how EMSTAC has supported your activities... 



19. How often do you have contact with EMSTAC or your TA liaison? (CONTACT, VISITS) 



20. During the current year how many times has your TA Liaison made a site visit? (Strateeies I and 
II only) 
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21. Please rank the following types of interactions with EMSTAC or your TA liaison, from the most 
common type to the least common. (If not applicable, write N/A) (SUPPORT) 

I requested information from the TA Liaison 

Catching Up/ Update - touching base 

Administrative 

The TA Liaison requests data from me 

Other: Please indicate 



22. What aspect of EMSTAC’s support has been most helpful in facilitating technical assistance in 
your district? (Training, access to researchers, materials sent, website, etc. . .) (IMPACT) 



23. On a scale of 1 to 5, 1 being Not Helpful, 5 being Very Helpful, how helpful has EMSTAC or 
your TA Liaison been in: (IMPACT) 



Not 

Helpful 

Problem identification/needs assessment 1 2 3 

Identifying research based materials 1 2 3 

Assisting LA in coordinating TA events 1 2 3 

Facilitating support and follow-up 1 2 3 

Motivating - moving the process 1 2 3 



Very 

Helpful 

4 5 

4 5 

4 5 

4 5 

4 5 



24. Have you found the EMSTAC listserv (LA Exchange) to be helpful? Why or why not? 
(IMPACT) 



25. Have you participated in any of the EMSTAC sponsored Chat Events? If yes, do you feel that the 
chat events have been beneficial? How? (IMPACT) 



26. Have you had occasion to enter either the public or private part of our website? Which? Do you 
believe that the EMSTAC website provides information that is useful and useable? (IMPACT) 
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27. Have you requested or been sent any of EMSTAC’s written products such as “How to select an 
intervention?” If yes, how would you rate the quality and usefulness of the information included 
in the products you have received? (IMPACT) 



28. Based upon your experience during the current year how important do you feel the support 
provided by EMSTAC or your TA Liaison was to you accomplishing your goals as a Linking 
Agent? (IMPACT) 



29. Do you have any suggestions for improving the support provided by EMSTAC? 
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EMSTAC EVALUATION 
LINKING AGENT INTERVIEW FORM II 
SPRING 2002 



This protocol should be used to guide your interview with Linking Agents and/or districts 
that have previously been evaluated by EMSTAC. If this is the first time this district has been 
evaluated please use Linking Agent Interview FORM I. There may be times when certain questions 
are inappropriate, irrelevant, or redundant. For example, although in their second or third year 
working with EMSTAC, it is possible that some districts will still be in the initial stages of 
implementing an intervention. In such cases these questions may be skipped because they are 
addressing issues of scaling up, sustainability and renewal. 



SAY: As you are likely aware, we are in the process of attempting to determine the most effective 
ways of providing technical assistance to school districts. Because your role as Linking 
Agent is seen as a critical element within the EMSTAC model, we wanted to spend about 30 
or 40 minutes getting your impressions about the Linking Agent role, your school district, 
and the type of support that EMSTAC has provided to you. 

Delivery of Technical Assistance and the Technical Assistance Environment (Scaling Up) 

1 . Since last year or since our previous visit has implementation or use of 

(identified intervention) expanded to additional classrooms or school 

buildings? (HAVELOCK; 12, 17, 18, 19) 



AND/OR 

2. Since last year or since our previous visit have new components of the old program been 
implemented? (HAVELOCK; 12, 17, 18, 19) 
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3. How have you attempted to “spread the word” or expand knowledge and interest about this 
intervention throughout your district? (HAVELOCK; 12, 17, 18, 19) 



Sustaining the Intervention 

4. How have you attempted to support those classrooms or buildings that were originally involved 
in this initiative so that they might continue to use the intervention? (COLLAB; 17, 18, 19) 



5. Have building-level or district level leaders made the intervention a part of the school’s or 
district’s future priorities? For example, is this initiative seen as a way to support the district’s 
current and future goals? (BLDGLEAD, DISTLEAD, COLLAB; 11, 12, 18) 



6. Have financial resources been allocated to this initiative so that it can be sustained in the future? 
(RESOURCE, EXTERNAL; 18) 



Renewal or Identifying a New Need or Intervention 



Please note that prior to conducting this interview you will be aware as to whether the district has 
identified a new need or intervention. If the district has not yet identified a new need or intervention then 
the following questions should not be asked. 

Please identify new need . 



7. How has your previous work with EMSTAC influenced your decision to identify and/or select a 
new intervention? In what way? (IMPACT) 
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8. In order to identify the new need that your district is now addressing, did you facilitate the completion 
of a formal needs assessment? Please explain.(NEED, MOTIVE; 3, 4, 5) 



9. Have formal, written technical assistance goals been established (or some type of written plan)? 
(NEED; 6) 



10. Please briefly describe the technical assistance activities you have coordinated (regarding the new 
need) that specifically address the district’s identified areas of need. (7) 



11. Has an evaluation plan been developed in order to determine the impact of the intervention? (13) 



12. (If not already answered) Do you have external sources of technical assistance besides EMSTAC to 
support this initiative? Please describe. (RESOURCE, EXTERNAL) 



13. Could you estimate how much time have you spent working with district administrators, school 
administrators, and teachers this year on this project? (Probe for % of time) 

Percent of time 

District administrators (DISTLEAD, COLLAR) 

School administrators (BLDGLEAD, COLLAR) 

Teachers (COLLAR) 
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14. On the scale below, please rate the extent to which individuals at the district and school level are 
open and willing to using external technical assistance related to providing services for children 
with special needs: 

Not Open to Change Very Open to Change 

1 2 3 4 5 



Rating 

District personnel (DISTOPEN) 

School personnel (SCHOPEN) 



15. What could your school district do to support you as a Linking Agent in future years? 
(COLLAR) 



16. Would you like to continue being a Linking Agent? Why or why not? 



SAY: Now we would like to find out how EMSTAC has supported your activities... 

17. How often do you have contact with EMSTAC or your TA Liaison? (CONTACT, VISITS) 



18. During the current year how many times has your TA Liaison made a site visit? (Strateeies I and 
II only) 
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19. Please rank the following types of interactions with EMSTAC or your TA liaison, from the most 
common type to the least common. (If not applicable, write N/A) (SUPPORT) 

I requested information from the TA Liaison 

Catching Up/ Update - touching base 

Administrative 

The T A Liaison requests data from me 

Other: Please indicate 



20. What aspect of EMSTAC ’s support has been most helpful in facilitating technical assistance in 
your district? (Training, access to researchers, materials sent, website, etc.. .) (IMPACT) 



21. On a scale of 1 to 5, 1 being Not Helpful, 5 being Very Helpful, how helpful has EMSTAC or 
your T A Liaison been in: (IMPACT) 



Not Very 

Helpful Helpful 



Problem identification/needs assessment 
Identifying research based materials 
Assisting LA in coordinating T A events 
Facilitating support and follow-up 
Motivating - moving the process 



1 2 3 4 

1 2 3 4 

1 2 3 4 

1 2 3 4 

1 2 3 4 



5 

5 

5 

5 

5 



22. Have you found the EMSTAC listserv (LA Exchange) to be helpful? Why or why not? 
(IMPACT) 



23. Have you participated in any of the EMSTAC sponsored Chat Events? If yes, do you feel that the 
chat events have been beneficial? How? (IMPACT) 
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24. Have you had occasion to enter either the public or private part of our website? Which? Do you 
believe that the EMSTAC website provides information that is useful and useable? (IMPACT) 



25. Have you requested or been sent any of EMSTAC’s written products such as “How to select an 
intervention?” If yes, how would you rate the quality and usefulness of the information included 
in the products you have received? (IMPACT) 



26. Based upon your experience during the current year how important do you feel the support 
provided by EMSTAC or your TA Liaison was to you accomplishing your goals as a Linking 
Agent? (IMPACT) 



27. Do you have any suggestions for improving the support provided by EMSTAC? 
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EMSTAC EVALUATION 
“OTHER” INTERVIEW SPRING 2002 



This protocol is to be used with individuals other than teachers who have worked with the 
Linking Agent on the targeted intervention program. These individuals may include, but are not 
limited to, district level administrators, building-level administrators, student support personnel, 
parents, or community members. 

Please be aware that some of the questions contained herein may not be appropriate for the 
person being interviewed. Additionally, depending on the progress that the district has made during 
the current year, some of the questions may not be applicable at this time. Thus, as the interview 
unfolds, the interviewer may chose to skip questions that the interviewee had no knowledge of or 
could not answer at this time. It is not expected that every question will be asked in all 
circumstances. For that reason this interview should not be considered a standardized measure with 
“standardized operating procedures.” 



SAY: We would like to start today by providing you with a little bit of background about our 

Center. As you might already be aware, our Center is a funded project through the United 
States Department of Education, Office of Special Education Programs (OSEP). Essentially, 
the goal or mission of our Center is to work with local school districts in assisting them to 
address needs that they have identified particularly those needs that relate to educating 
students with disabilities. During the course of the current year we have been working 

closely with (name of Linking Agent) to address the needs identified by your school 

district. The purpose of our interview is to learn about the work that you have been engaged 

in with (name of Linking Agent). In addition, we would also like your impressions 

about the types of support that (name of Linking Agent) has provided to this project. 

Prior to beginning the interview please note whether this administrator was interviewed during a previous 
evaluation cycle by EMSTAC. If yes, please determine whether you will be re-visiting the “old” need or 
whether you will be discussing a “new” need. Questions with asterisks ** can be skipped if you are 
discussing an “old” need. 
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Interviewee’s Name and Position: 



The following questions can be asked of all administrators regardless of whether they are 
building based or district based. 

1. What is your role with the “new” initiative? or How, if any, has your role with the “old” initiative 
changed? (Probe: how actively have you been involved with selecting and implementins the 
initiative?') (DISTLEAD; BLDGLEAD) 



2. Are you familiar with how this particular program was chosen? What were the criteria for 
choosing this program? (Probe for locus of change and motivation for addressing the need) 
(LOCUS, MOTIVE, STATUS; 3, 4, 5, 6) 



3. How did you learn about this particular program prior to deciding on its implementation? (3, 8, 9, 

10 ) 



4 **What are the goals of this project? (6) 



5. In terms of (identified need) what internal and external resources, other than (name of Linking 
Agent) & EMSTAC, do you have available and are those resources helpful? (Probe for both 
internal and external resources) (EXTERNAL, RESOURCE, COLLAR) 



6. (If not already answered) Did you have to secure outside sources of funding (outside of the 
district budget allocations) in order to start the program? How did you go about obtaining 
these funds? (EXTERNAL, RESOURCE; 7) 
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EXHIBIT A-23: 

District 24 Technical Assistance 
Delivery Model, 2001-2002 
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7. (If not already answered) Do you have external sources of technical assistance besides 
EMSTAC to support this initiative? Please describe. (EXTERNAL; 7) 



8. Could you estimate how much time have you spent working with (Linking Agent’s Name) this 
year on this project? (BLDGLEAD; DISTLEAD; 7, 10) 



9 **During the year were you aware that (Linking Agent’s Name) was collaborating with our 
Center? What did he/she tell you about our Center? (BLDGLEAD, DISTLEAD, COLLAB; 8, 
9,10) 



10. In working with you, what type of support did (Linking Agent’s Name) provide? Did (Linking 
Agent’s Name) play a role in providing you with information about new research-based 
programs or interventions? (Probe for how much collaboration is soins on) (COLLAB: 9, 10) 



\\. In addition to the initiative we are currently discussing, is your district/school currently and 
actively implementing initiatives or reforms to improve outcomes for children in special 
education? Please describe. (CURRENT) 



12. In the past five years, has your district/school adopted initiatives or reforms to improve outcomes 
for children in special education (other than the current ones mentioned above)? Please describe. 
(PAST) 
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13. What initiatives or goals are currently of greatest priority in your district? (CURRENT) 



14. Is scaling up the use of this intervention considered a priority either within your school or the 
district? Why or why not? (7, 8, 12, 17) 



15. Do you believe that this intervention is critically important in helping the district meet goals that 
are part of the strategic or school improvement plan? If yes, what plans are in place to ensure that 
the program will be sustained in future years? (7, 8, 18, 19) 



In addition to the above questions, principals or building-level administrators should be asked the 
following: 



16. **Have teachers in the school actually begun to implement new instructional or management 
strategies? (IMPACT) 



17. What strategies or supports have been put into place to expand the use of the intervention within 
your building? (12, 17) 



18. Could you describe some of the ways that (the Linking Agent) specifically supported the staff in 
their efforts to implement program strategies? (Probe for how much collaboration is soins on) 

(COLLAR) 



O 



B-16 



American Institutes for Research 



19. How are you (or how is the team) evaluating whether the program is actually working? What 
have you found? Q3) 



20. What barriers have you encountered in your attempts to go forward with this project? 



21. What support have you provided to (Linking Agent’s Name) to facilitate their work on this 
initiative? (Probe for type and extent of support) (DISTLEAD. BLDGLEAD; 8) 



22. Is someone other than the Linking Agent involved in problem-solving issues that arise regarding 
the intervention? Do you feel that this intervention could last even if the Linking Agent were no 
longer to provide ongoing support? (COLLAB; 18) 



23. What support has (district or school) provided to (Linking Agent’s Name) to facilitate their work 
on this initiative? (Probe for type arid extent of support) (DISTLEAD. BLDGLEAD, 
COLLAB; 8) 



24. On a scale of 1 to 5 (with 1 being “not at all open” and 5 being “very open”), how open has the 
school been to the idea of using outside technical assistance? (SCHOPEN) 



25. On a scale of 1 to 5 (with 1 being “not at all open” and 5 being “very open”), how open has the 
district been to the idea of using outside technical assistance? (DISTOPEN) 
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EMSTAC EVALUATION 
TA LIAISON INTERVIEW SPRING 2002 

The purpose of this interview is to increase the Center’s understanding of the many 
components needed to provide quality technical assistance. In particular, this interview is intended to 
provide you with an opportunity to reflect upon your experiences this year and to allow you to 
expand upon the information that you documented in the TA Liaison communication logs. Finally, it 
is hoped that this information will allow the Center to improve the way it provides support to TA 
Liaisons. Please answer the following questions as honestly as possible using any notes or other 
documents that you may have. 

Understanding the TA Liaison Role 

1 . Describe the tasks associated with your role as a TA Liaison. 



2. What do you believe are the most important skills that a TA Liaison must have in order to be 
effective? Examples? (15) 



The TA Liaison and EMSTAC 

3. What aspects/resources of EMSTAC were most helpful in allowing you to meet your 

responsibilities as a TA Liaison? Do you feel that additional resources or supports are needed? If 
so what additional supports/resources? (15) 
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For the following sections, please answer all of the questions for each district you work with as TA 
Liaison. Also, please note any differences or similarities across strategies. 

Working with Local School Districts 

4. What were your main goals in working with your district(s)? Did these goals change during the 
course of the year? (15) 



5. In your opinion, what aspects of the support provided by EMSTAC have been most beneficial to 
the school district in their efforts to implement technical assistance activities? What aspects have 
been least beneficial? (IMPACT; 15) 



6. How was this support delivered? (SUPPORT) 



7. Are there any unique characteristics/circumstances of your Linking Agent/district that either 
helped or hindered your work with the district this year? 



8. Which EMSTAC resources and supports are being accessed/requested and how frequently? 



Outcomes 



9. Where in the change process (Havelock’s model) was your district at the time of Linking 
Agent training or the last evaluation point (end of last year)? Where is the district now? 
(HAVELOCK) 
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10. Did your district choose the initiative before working with EMSTAC or as a result of the 
relationship with EMSTAC? (STATUS) 



1 1 . Please describe some of the technical assistance activities that have occurred as a result of 
EMSTAC’s support. (IMPACT) 



12. Who, if anyone (EMSTAC/the district), is evaluating the initiatives/activities occurring in the 
district? If evaluation is being conducted, what is being found? What types of evidence (data) 
are being gathered (teacher surveys, test score data, etc.)? 



13. Are teachers actually engaging in new teaching behaviors? To what degree is this facilitated 
by support being provided by the Linking Agent? Do teachers believe the support they 
receive is important in helping them implement new practices? Can they describe examples? 



State-Level Initiatives and Strategic Partnerships 

14. If you have worked with state representatives or strategic partners, please comment on your 
experience. Include history of relationship, benefits of working with state/strategic partners, 
challenges associated with working with state/strategic partners, and your impressions of how 
districts view these partners. 



Time Allocation 

14. Please indicate, in the space below, how much time you have spent on the following three 
areas only, since the end of June, 2001, for each district that you are the TA Liaison for: 
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Note: Training Activities include actual training as well as coaxing linking agents through the 
training (for Strategy III districts you work with). TA Support Activities include product 
development, but not evaluation activities. 



District 1 Name: 



(Strategy ) 

Recruitment Activities: % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) 

Training Activities: : % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) % of your total time 

TA Support Activities: : % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) % of your total time 



District 2 Name: 



(Strategy 



Recruitment Activities: : % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) 

Training Activities: : % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) 

TA Support Activities: : % of your total time in Year 1 

% of your total time in Year 2 

% of your total time in Year 3 

% of your total time in Year 4 

% of your total time in Year 5 (so far) 

District 3 (if applicable, fill in as with Districts 1 and 2 above) 
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Reflection Questions on EMSTAC’s 5 Years 

16. Do you feel you have been a usefiil/positive influence to your districts? Please provide examples. 



17. How would you do things differently? 



18. Are there any further comments you would like to add regarding your experience with 
EMSTAC? Things that worked well? Ways that could be improved? 
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EMSTAC EVALUATION 
TEACHER INTERVIEW SPRING 2002 



This protocol should be used to interview each teacher that you meet with. There may be 
times when certain questions are inappropriate, irrelevant, or redundant. In such cases these questions 
may be skipped. 



SAY: and I are here from Washington D.C. We work on a project called the Elementary and 

Middle Schools Technical Assistance Center. The Center is an Office of Special Education 
Programs (OSEP) funded project through the United States Department of Education. 
Essentially, the goal or mission of our Center is to work with local school districts in 
assisting them to address needs that they have identified particularly those needs that relate 
to educating students with disabilities. During the course of the current year we have been 

working closely with (name of Linking Agent) to address the needs identified by your 

school district. 

The decision to work with your district was simply based upon the fact that the district 
agreed to work with us. At the beginning of the year we made contact with the director of 
special education in your district to see if there might be any interest in working with us. Our 
decision to contact your district in particular was solely in an attempt to find districts from 
various regions across the country, so your district as well as many others in this area was 
initially contacted. The purpose of our interview is generally to ascertain your impressions 

about the program that you have been involved in implementing this year. In 

addition, we would also like to learn a little more about your particular school so we will be 
asking some questions about that as well. 

Delivery of Technical Assistance and the Technical Assistance Environment 

Please indicate which intervention this teacher is implementing and whether this is the first 
need/intervention or subsequent need/intervention being addressed. 



1. When did you begin implementing this intervention? (STATUS) 



2. Why did you choose to get involved with this project? (LOCUS, MOTIVE) 
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3. In preparation for the implementation of , did you receive any training or participate in any 

professional development activities? Please be specific. (Probe for evidence of ongoing 
collaboration) (COLLAB^ 



4. What type of support was provided after the training was conducted? Who provided this support? 
(Probe for evidence of ongoing collaborationJfEXTElRNALi, COLLAB; 8) 



5. How has the school supported the implementation of the initiative? (BLDGLEAD; DISTLEAD; 
8,9) 



Adopted Program/Intervention 

6. How easy was it to implement the program? Did it require much planning time? 



***The next three questions should only be asked of teachers who have been involved in 
implementing the intervention for one school year or longer. 



7. Has this intervention become a routine part of your teaching? Explain. (18) 



8. How has the Linking Agent continued to support you in your efforts to implement this 
intervention? (COLLAB; 10,11,18) 
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9. Is there a plan in place to monitor how successful this intervention is? (13) 



Outcomes 

10. What changes, if any, would you make to for next year? 



11. How do you think has impacted your students (learning, outcomes, achievement)? 

Impacted you as a teacher (ideas, teaching philosophy)? (IMPACT) 



12. How has impacted your awareness of the learning needs of students with disabilities and 

does this program make instruction more responsive to those needs? (IMPACT) 



13. What are the overall weaknesses of ? 



14. What are the most critical aspects of this program that make it effective? 



15. Is there anything else you would like to share with me? 




B-25 



387 



American Institutes for Research 



Spring 2002 
Teacher Survey 




388 



EMSTAC EVALUATION 
TEACHER SURVEY SPRING 2002 



This teacher survey is designed to collect information about three areas important to EMSTAC ’s 
evaluation: 

(a) your perceptions of the delivery of the technical assistance; 

(b) your opinion regarding feasibility of the intervention; and 

(c) your satisfaction regarding intervention outcomes. 

During the current school year you’ve had an opportunity to implement new practices and 
techniques that improve outcomes for students with disabilities. As a “behind the scenes” partner 
with your school district, the Elementary and Middle Schools Technical Assistance Center 
(EMSTAC) is interested in your opinion regarding these practices. In particular, we would like to 
know how you’ve used the intervention, your thoughts on its effectiveness, and your perceptions of 
the training and support the Linking Agent and/or consultants have provided. Finally, we would like 
to know if you feel that the use of these strategies had an impact on your students based on your 
judgments and on classroom and other data. 

For each item that follows please respond by circling one of the following choices: 

Strongly Somewhat Neither Somewhat Strongly 
Disagree Disagree Agree nor Agree Agree 

Disagree 

Delivery of Technical Assistance and the Technical Assistance Environment 

1 . I agree that the identified need 1 2 3 4 5 

should be a priority in our 

school/district. 

2. I feel that I know how or why this 1 2 3 4 5 

particular program was selected 

by the district and/or the Linking 
Agent. 

3. I feel that the intervention or 1 2 3 4 5 

program our district chose to 

implement is the most suitable for 
our needs and goals. 
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Strongly 


Somewhat 


Neither 


Somewhat 


Strongly 




Disagree 


Disagree 


Agree nor 
Disagree 


Agree 


Agree 


4. The Linking Agent and/or other 
staff or consultants provided me 
with training on techniques 
associated with the program, prior 
to my implementing them in the 
classroom. 


1 


2 


3 


4 


5 


5. I believe that the training 

provided me with opportunities to 
develop new skills. 


1 


2 


3 


4 


5 


6. I had an opportunity to practice 
these skills prior to implementing 
the intervention. 


1 


2 


3 


4 


5 


7. The Linking Agent or related 
project staff have provided 
feedback on my performance of 
these skills. 


1 


2 


3 


4 


5 


8. I felt reasonably confident that I 
could implement the intervention 
by the time I was expected to use 
these strategies 


1 


2 


3 


4 


5 


9. Generally, I am comfortable using 
this program in my classroom. 


1 


2 


3 


4 


5 


10. I know I can go to the Linking 
Agent or related staff if I have 
questions about the program. 

Feasibility 


1 


2 


3 


4 


5 


1 1 . The program was easy to 
implement. 


1 


2 


3 


4 


5 


12. The intervention philosophy is 
compatible with the norms and 
values of my school. 


1 


2 


3 


4 


5 


13. The philosophy of this program is 


1 


2 


3 


4 


5 



consistent with my beliefs about 
teaching. 
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Strongly 

Disagree 


Somewhat 

Disagree 


Neither 
Agree nor 
Disagree 


Somewhat 

Agree 


Strongly 

Agree 


14. A significant amount of planning 
time was needed to initially 
implement the program. 


1 


2 


3 


4 


5 


15. The effort required to implement 
the program is worth the results. 


1 


2 


3 


4 


5 


16. The program allows me to make 
modifications that meet my 
classroom needs. 


1 


2 


3 


4 


5 


17. This program requires me to 
prepare my lessons much 
differently than how I used to. 


1 


2 


3 


4 


5 


Outcomes 












1 8. I have learned new techniques to 
teach my students as a result of 
my involvement with the 
program. 


1 


2 


3 


4 


5 


19. This program has had some 
influence on how I think about 
teaching students with special 
needs. 


1 


2 


3 


4 


5 


20. This program has increased my 
awareness of the learning needs of 
students with disabilities. 


1 


2 


3 


4 


5 


21.1 believe that the program is 
effective in making instruction 
more responsive to student needs. 


1 


2 


3 


4 


5 


22. This is a program that I would 
recommend to others. 


1 


2 


3 


4 


5 


23. I would still support the adoption 
of this program now that I have 
had an opportunity to implement 
it. 


1 


2 


3 


4 


5 


24. I believe that this program 


1 


2 


3 


4 


5 



provides students with special 
needs more opportunities to learn. 
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Strongly 

Disagree 



Somewhat 

Disagree 



Neither 
Agree nor 
Disagree 



Somewhat 

Agree 



Strongly 

Agree 



25. Students seem to enjoy the 1 2 3 4 5 

learning activities associated with 

this program. 

26. I believe that the implementation 1 2 3 4 5 

of this program has had a positive 

impact on student achievement. 

27. I have data that illustrates the 1 2 3 4 5 

program’s positive impact on 

student achievement. 

28. I see myself using these 1 2 3 4 5 

techniques five years from now. 



Please answer the following questions to the fullest extend possible: 

29. Describe the training and/or support (referenced in #3 above) that you have received from the 
Linking Agent and/or other staff and consultants on this project? Please be specific. 
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30. Do you feel this support has helped you implement this intervention in your classroom? Please 
explain why or why not. 



31. If you had the opportunity to select a new intervention to use in your classroom next fall, 
regarding the current or another topic, how would you go about doing so? Please be specific. 



***Please seal your completed survey in the envelope provided and deliver this sealed envelope 
to your Linking Agent. You will be thanked for your time with a Barnes and Noble gift 
certificate, to be distributed by your Linking Agent upon completion of this survey. Thank you 
for your participation and cooperation. 
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APPENDIX C: 



EMSTAC PRESENTATIONS AND PRODUCTS 



Chat Events 

Artiles, A. (Facilitator). Culturally responsive instruction. Chat event. Washington, DC: Elementary 
and Middle Schools Technical Assistance Center, November, 2001. 

Ball, W. (Facilitator). Conducting a needs assessment at the local level. Chat event. Washington, 

DC: Elementary and Middle Schools Technical Assistance Center, May, 2001. 

Beverly, C. (Facilitator). University partnerships: Benefits and process. Chat event. Washington, 

DC: Elementary and Middle Schools Technical Assistance Center, October, 2001. 

Dimino, J. (Facilitator). Literacy interventions for elementary and middle schools. Chat event. 

Washington, DC: Elementary and Middle Schools Technical Assistance Center, May, 2001. 

Ford, B. (Facilitator). The Importance of Home-School-Community Partnerships in Minority 
Communities. Chat event. Washington, DC: Elementary and Middle Schools Technical 
Assistance Center, February, 2002. 

Gersten, R. (Facilitator). Instructional strategies for English language learners. Chat event. 

Washington, DC: Elementary and Middle Schools Technical Assistance Center, September, 
2000. 

Gillins, S. (Facilitator). Spring forward: First steps as a new Linking Agent. Chat event. Washington, 
DC: Elementary and Middle Schools Technical Assistance Center, March, 2001. 

Holodak, N. (Facilitator). Integrating Mathematic Concepts into Your Elementary and Middle School 
Curriculum. Chat event. Washington, DC: Elementary and Middle Schools Technical 
Assistance Center, April, 2002. 

Llewellyn, V., & Thomas, D. (Facilitators). Life as an EMSTAC Linking Agent. Chat event. 

Washington, DC: Elementary and Middle Schools Technical Assistance Center, January, 
2001. 

Mason, A. (Facilitator). PAM: Prereferral assessment model. Chat event. Washington, DC: 
Elementary and Middle Schools Technical Assistance Center, July, 2001. 

Mattingly, L. (Facilitator). Reducing behavioral referrals, suspensions, and expulsions: The use of 
social skills training and other behavioral interventions in the classroom. Chat event. 
Washington, DC: Elementary and Middle Schools Technical Assistance Center, August, 

2001. 

Mesmer, E., & Woodruff, D. (Facilitators). Behavioral interventions and strategies. Chat event. 

Washington, DC: Elementary and Middle Schools Technical Assistance Center, May, 2000. 
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Ortiz, A. (Facilitator). Preventing inappropriate referrals. Chat event. Washington, DC: Elementary 
and Middle Schools Technical Assistance Center, December, 2001. 

Townsend, B., & Webb-Johnson, G. (Facilitators). Culturally Responsive Classroom Management. 
Chat event. Washington, DC: Elementary and Middle Schools Technical Assistance Center, 
March, 2002. 

Tracey, D. (Facilitator). Using co-teaching and teaming strategies to facilitate inclusion for students 
with disabilities. Chat event. Washington, DC: Elementary and Middle Schools Technical 
Assistance Center, July, 2001. 



Internal Support Documents 

Corwin, H., Ruedel, K., Diamond, C., & Shanley, J. (2001). EMSTAC staff manual. Prepared for the 
Elementary and Middle Schools Technical Assistance Center, Washington, DC. 

Margolin, J. (2001). EMSTAC database manual. Prepared for the Elementary and Middle Schools 
Technical Assistance Center, Washington, DC. 

Shanley, J. (2000). EMSTAC materials and product development team manual. Prepared for the 
Elementary and Middle Schools Technical Assistance Center, Washington, DC. 



Presentations 

Acosta, B. (2000, October). Math the way your teacher never taught you: Multicultural mathematics. 
Presentation conducted at the CEC/DDEL Symposium on Culturally and Linguistically 
Diverse Exceptional Learners, Albuquerque, NM. 

Acosta, B. (2001, April). One right way? What culturally diverse students can teach us about math. 
Presentation conducted at the Council for Exceptional Children Annual Convention and 
Exposition, Kansas City, MO. 

Acosta, B. (2002, March). Mathematics instruction for English language learners with disabilities. 
Presentation conducted at the National Association for Bilingual Education Conference, 
Philadelphia, PA. 

Acosta, B. (2002, April). Instructional Strategies for Teaching Mathematics to Culturally and 

Linguistically Diverse Students with Disabilities. Presentation conducted at the Council for 
Exceptional Children Convention, New York, NY. 

Acosta, B. & Brown, C. (2000, February). Designing a portfolio assessment for English language 
learners. Presentation conducted at the National Association of Bilingual Education 
Conference, San Antonio, TX. 
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Acosta, B. & Brown, C. (2001, February). Alternative assessment for English language learners. 
Presentation conducted at the National Association of Bilingual Education Conference, 
Phoenix, AZ. 

Acosta, B. & Killos, L. (1999, June). Training on the process of change. Training conducted on 

behalf of the Elementary and Middle Schools Technical Assistance Center for a participating 
district. 

Acosta, B. & Killos, L. (2000, April). Designing a portfolio assessment for English language 

learners. Poster presented at the Council for Exceptional Children Annual Convention and 
Exposition, Vancouver, BC. 

Acosta, B., Killos, L. & Thomas, D. (1999, November). Becoming a change agent for research- 

based change. Presentation conducted at the Collaboration Across Borders Conference, Las 
Cruces, NM. 

Acosta, B. & O’Cummings, M. (2002, January). Training on the process of change. Training 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Acosta, B. & Thomas, D. (2000, June). Implementing effective research-based change. Presentation 
conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Davis, C., Trent, S., Kea, C., Artiles, A.,Rodriguez, D., Jordan, K., Hampton, S., Froelich, K., 

Chavez, J., Voltz, D., & Davis, J. (2002, April). Culturally sensitive research sponsored by 
the Center of Minority Research in Special Education. Panel discussion conducted at the 
Council for Exceptional Children Annual Convention and Exposition, New York, NY. 

Diamond, C., & Ruedel, K. (2000, November). Using distance education as a training vehicle: 

Research to practice. Presentation conducted at the 43rd Annual Conference of the Virginia 
State Federation Council for Exceptional Children, Dulles, VA. 

Diamond, C., & Schwartz, A. (2002, April). All children can read: Using adapted books to motivate 
student literacy. Poster presented at the Council for Exceptional Children Annual Convention 
and Exposition, New York, NY. 

Diamond, C., & Schwartz, A. (2002, April). Making a match: What to consider when selecting an 
intervention or program. Presentation conducted at the Council for Exceptional Children 
Annual Convention and Exposition, New York, NY. 

Dwyer, K. & Mesmer, E. (1998, September). Early warning - Timely response: A guide to safe 

schools. Presentation conducted at the International Council for Adolescents with Emotional 
and Behavioral Disorders, Bethesda, MD. 

Ervin, L. & Welch, C. (2001, October). Overview of disproportionality and its implications for 
African American children and youth. Presentation conducted at the 100 Black Men of 
America for the Wimberly Initiative Bi-Annual Meeting, Las Vegas, NV. 
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Gillins, S. & Welch, C. (2001, October). EMSTAC: A problem solving approach for identifying 
effective resources and interventions for disproportionality. Presentation conducted at the 
100 Black Men of America for the Wimberly Initiative Bi-Annual Meeting, Las Vegas, NV. 

Hamilton, J. (1998, February). Moving research into practice in special education. Panel discussion 
conducted at the National Information Specialists Meeting, Washington, DC. 

Hamilton, J., Ball, W., & Zorfass, J. (1999, June). Lessons learned in providing technical assistance 
at the local level. Presentation conducted at the Office of Special Education Programs 9'*’ 
Annual Technical Assistance and Dissemination Conference, Washington, DC. 

Hamilton, J., Dailey, D., Carl, B., & Ritter, S. (2002, April). Is Your School Improving Outcomes for 
Students with Disabilities: Guesswork or Science? Presentation conducted at the Council for 
Exceptional Children Convention, New York, NY. 

Hamilton, J., & Mclnemey, M. (1999, October). Moving research into practice in special education. 
Presentation conducted at the American Institutes for Research Board Presentation, 
Washington, DC. 

Hamilton, J., & Mclnemey, M. (2001, April). EMSTAC: Providing inside-outside support for new 
practices. Presentation conducted at the Council for Exceptional Children Annual 
Conference and Exposition, Kansas City, MO. 

Hamilton, J., Mclnemey, M., Thomason, J., & Dimino, J. (1999, April). National technical 

assistance needs: Strategies to improve the picture. Presentation conducted at the Annual 
Council for Exceptional Children Conference, Charlotte, NC. 

Hamilton, J., Mclnemey, M., Thomason, J., & Dimino, J. (2000, April). Reforming schools in an era 
of increased accountability: A professional development model. Presentation conducted at the 
Annual Council for Exceptional Children Conference, Vancouver, B.C. 

Hamilton, J., Mclnemey, M., Woodmff, D., Thomason, J., & Dimino, J. (2001, April). Achieving 
accountable schools through Linking Agents: An insider-outsider professional development 
model. Presentation conducted at the Annual Council for Exceptional Children Conference, 
Kansas City, MO. 

Hamilton, J., & Pisacane, K. (1999, April). Meeting the technical assistance needs of elementary and 
middle school students. Poster presented at the Annual Council for Exceptional Children 
Conference, Charlotte, NC. 

Hamilton, J., & Zorfass, J. (1998, June). Technical assistance lessons at the local level. Presentation 
at the Office of Special Education Programs 8* Annual Technical Assistance and 
Dissemination Conference, Washington, DC. 

Havelock, R., & Ruedel, K. (2002, April). A change agent in special education: A new role — A new 
opportunity. Presentation conducted at the Council for Exceptional Children Annual 
Convention and Exposition, New York, NY. 
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Humphrey, L., Feam, K. & Welch, C. (2001, November). Addressing overrepresentation of African 
American and other students through promising practices. Presentation conducted at the 
National Alliance of Black School Educators Annual Conference, Los Angeles, CA. 

Killos, L., Nishi, L, & Ruedel. K. (2001, March). Training on the process of change. Training 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Killos, L. & Ritter, S. (2001, April). What is left to learn? Using classroom based assessments in 
early childhood literacy programs. Presentation conducted at the Council for Exceptional 
Children Annual Conference and Exposition, Kansas City, MO. 

Lyons, E., Mesmer, E. & Crawford, K. (1999, April). From elementary to middle school: A 
longitudinal analysis of a nationally recognized school reform project. Presentation 
conducted at the National Association of School Psychologists, Las Vegas, NV. 

Mclnemey, M., Nishi, L. & Shami, M. (1999, October). EMSTAC recruitment. Presentation 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Mclnemey, M. & Woodruff, D. (1998, April). Overcoming "parallel play”: Developing new systems 
to promote family and professional collaboration. Presentation conducted at the Council for 
Exceptional Children Annual Conference and Exposition, Minneapolis, MN. 

Mclnemey, M., Woodmff, D., [Margolin, J. & Ruedel, K. via videoconference] (2001, November). 

Effective partnerships to improve services for children. Presentation conducted at the Parents, 
Inc. Pathways Conference, Anchorage, AK. 

Mclnemey, M., Woodmff, D. & Walsh, K. (2000, June). Providing technical assistance at the local 
level: Lessons learned. Presentation conducted at the Office of Special Education Programs 
10*'’ Annual Technical Assistance and Dissemination Conference, Washington, DC. 

Mesmer, E. (2001 , November). Developing guidelines for the assessment of students suspected of 
having an emotional disturbance - Part I. Presentation conducted on behalf of the 
Elementary and Middle Schools Technical Assistance Center for a participating district. 

Mesmer, E. (2002, January). Developing guidelines for the assessment of students suspected of 
having an emotional disturbance - Part II. Presentation conducted on behalf of the 
Elementary and Middle Schools Technical Assistance Center for a participating district. 

Mesmer, E. (2002, Febmary). Congruent use of behavioral interventions across classroom settings: 
Implications for the development of interventions for students with emotional/behavioral 
disorders within inclusive environments. Presentation conducted at the National Association 
of School Psychologists Annual Convention, Chicago, IL. 

Mesmer, E., & Diamond, C. (2000, November). Improving student outcomes: The Elementary and 
Middle Schools Technical Assistance Center. Presentation conducted at the State Meeting of 
Directors of Special Education, Kansas State Board of Education, Topeka, KS. 
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Mesmer, E., Diamond, C. & Welch, C. (2001, March). What is ED? And other behavioral issues. 
Presentation conducted on behalf of the Elementary and Middle Schools Technical 
Assistance Center at a Professional Development In-Service Training for a participating 
district. 

Mesmer, E. Nishi, L. & Shami, M. (1999, November). Training on the process of change. Training 
conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Mesmer, E., Nishi, L. & Shami, M. (2000, January). Training on the process of change. Training 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Mesmer, E., Ritter, S. & Carl, B. (2001, April). Can we bridge the gap? Evaluating Systemic Change 
Efforts at the Local Level. Presentation conducted at the Council for Exceptional Children 
Annual Convention and Exposition, Kansas City, MO. 

Mesmer, E., Shami, M., Nicholas, D. &Nizio, M. (2001, October) Collaboration for behavioral 
success: Linking local school districts to national resources. Presentation conducted at the 
Getting Positive About Supporting Student Behavior Conference, Detroit, MI. 

Mesmer, M., Shanley, J. & Woodruff, D. (1999, December). EMSTAC: Where we are, and where we 
are headed. Presentation conducted at the National Advisory Group Meeting, Detroit, MI. 

Mesmer, M., Shanley, J. & Woodruff, D. (2001, January). The Elementary and Middle Schools 

Technical Assistance Center (EMSTAC): A project overview. Presentation conducted for the 
staff of the American Institutes for Research, Washington, DC. 

Mesmer, E. & Woodruff, D. (1999, April). Using the stop and think problem solving process. 
Presentation conducted on behalf of the Elementary and Middle Schools Teclmical 
Assistance Center for a participating district. 

Mesmer, E. & Woodruff, D. (2001, April). Characteristics of effective school-based behavioral 
support programs. Poster presented at the Council for Exceptional Children Annual 
Convention and Exposition, Kansas City, MO. 

Mesmer, E. & Woodruff, D. (2001, October). The Elementary and Middle Schools Technical 

Assistance Center: Collaborating for success. Presentation conducted at the State Special 
Education Administrators Conference, Columbia, SC. 

Mesmer, E., Woodruff, D. & Sims, A. (2000, April). Strategies for developing school-based violence 
prevention and response plans. Presentation conducted at the Council for Exceptional 
Children Annual Convention and Exposition, Vancouver, BC. 

Nishi, L. & Shami, M. (2000, May). Training on the process of change. Training conducted on 

behalf of the Elementary and Middle Schools Technical Assistance Center for a participating 
district. 

Nishi, L., Shami, M., Javorsky, R. & Sams, S. (2001, April). Improving outcomes for students with 
disabilities: Collaborating to provide research-based practices in classrooms. Presentation 
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conducted at the Council for Exceptional Children Annual Convention and Exposition, 
Kansas City, MO. 

Nishi, L., Shami, M. Javorsky, S. & Sams, S. (2001, November). Collaboration: Teaming to provide 
technical assistance to local school districts to improve outcomes for students with 
disabilities. Presentation conducted at the Association for Education Service Agencies 
(AESA) Annual Conference and Exposition, Atlanta, GA. 

Nishi, L., Shami, M. Javorsky, S. & Sams, S. (2002, April). Collaboration: Teaming to provide 
technical assistance to local school districts. Presentation conducted at the Council for 
Exceptional Children Convention, New York, NY. 

Nishi, L., Shami, M. & Sams, S. (2001, October). EMST AC Recruitment. Presentation conducted on 
behalf of the Elementary and Middle Schools Technical Assistance Center for a participating 
district. 

Nishi, L., Shami, M. & Sams, S. (2002, January). Training on the process of change. Training 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for 
for a participating district. 

O'Cummings, M. (2002, April). Spreading the news: Using distance education to reach out to local 
school districts. Panel discussion conducted at the Council for Exceptional Children 
Convention and Exposition, New York, NY. 

O’Cummings, M. & Whittington, R. (2002, April). Alternative day program: Temporary educational 
placement for students with behavioral disabilities. Poster presented at the Council for 
Exceptional Children Convention and Exposition, New York, NY. 

Ruedel, K. & Diamond, C. (2001, October). A Guide for education personnel: Selecting an 

intervention or program. Paper presented at the 44th Annual Conference of the Virginia State 
Federation Council for Exceptional Children, Fredericksburg, VA. 

Shanley, J. (2000, March). Measuring outcomes of youth leadership programs: Using program 
evaluation principles effectively. Presentation conducted at the President’s Committee on 
Employment of People with Disabilities Annual Youth Leadership Meeting, Washington, 

DC. 

Shanley, J. (2000, June). Linking with EMSTAC to access research based resources. Presentation 

conducted on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Shanley, J. (2000, November). N on-traditional career paths for special education personnel: The 
new millennium brings new opportunities. Presentation conducted at the Teacher Education 
Division, Council for Exceptional Children Annual Meeting, Las Vegas, NV. 

Shanley, J. (200 1 , June). The Linking Agent approach as a community of practice to accomplish 

change at the local level. Presentation conducted at the Office of Special Education Programs 
1 1* Annual Technical Assistance and Dissemination Conference, Washington, DC. 



O 



C-7 



402 



American Institutes for Research 



Shanley, J., Pisacane, K. & Ritter, S. (2000, April). Technology in professional development: 
Building research practices into teaching through Web based training. Presentation 
conducted at the Council for Exceptional Children Annual Convention and Exposition, 
Vancouver, BC. 

Shanley, J., Rodriguez, C. & Brown, J. (2001, October). The use of adult learning theory to guide 

curriculum development and the use of interactive multimedia. Presentation conducted at the 
Cultural Competent Curriculum Module Project Kick-off Meeting, Washington, DC. 

Shanley, J. & Ruedel, K. (2001, April). Sharing our learning - Accessing research-based practices 
through the Web. Presentation conducted at the Council for Exceptional Children Annual 
Convention and Exposition, Kansas City, MO. 

Shanley, J. & Whittington, R. (2002, April). An alternative day program for students with emotional 
and behavioral disabilities. Presentation at the Council for Exceptional Children Convention 
and Exhibition, New York, NY. 

Shanley, J., Woodruff, D. & Mesmer, M. (2001, December). EMSTAC: Where we are, and where we 
are headed. Presentation conducted at the National Advisory Group Meeting, Washington, 
DC. 

Sims, A. (2000, February). Minority disproportionality in special education. Presentation conducted 
on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Sims, A., Smith, M. & Welch, C. (2000, October). A School based model to examine minority 
disproportionality in special education. Presentation conducted at the Council for 
Exceptional Children (CEC) Division for Culturally and Linguistically Diverse Learners 
(DDEL) Bi-Annual Meeting, Albuquerque, NM. 

Sims, A., Townsend, B., Welch, C. & Woodruff, D. (2000, October). School-wide discipline: 
Ensuring successful school experiences for all students. Presentation conducted at the 
Council for Exceptional Children (CEC) Division for Culturally and Linguistically Diverse 
Learners (DDEL) Bi-Annual Meeting, Albuquerque, NM. 

Sims, A., Welch, C., Adams, J.Q. & McGowan, D. (2001, June). Disproportionality in special 
education and the achievement gap: Common issues, shared solutions. Presentation 
conducted at the Policymaker Partnership Annual Meeting, Washington, DC. 

Sims, A., Welch, C. & Smith, M. (2001, February). A District-wide model to examine minority 
disproportionality: Raleigh retreat. Presentation conducted at the Annual Retreat for 
Administrators in North Carolina, Pinehurst, NC. 

Sims, A., Woodruff, D. & Mesmer, E. (2000, April). Alternative educational interim placements: 
What works. Presentation conducted at the Council for Exceptional Children Annual 
Convention and Exposition, Vancouver, BC. 

Welch, C. (1999, November). The ILIAD Partnership-Special education and the African American 
child: Solutions for the problem of disproportionate representation. Panel member at the 
National Alliance of Black School Educators (NABSE) Annual Conference, Nashville, TN. 
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Welch, C. (2000, Apriy. Exploring parents ’ experiences during transition: A catalyst for creating 
culturally sensitive paradigms. Paper presented at the Multicultural Education and Cultural 
Diversity Symposium, Hampton University, Hampton, VA. 

Welch, C. (2001, April). Culturally competent assessment. Panel member at the National Association 
of School Psychologists Annual Convention, Washington, DC. 

Welch, C. (2001, August). EMSTAC generated pre-referral interview instrument and effective 

interviewing techniques. Training conducted on behalf of the Elementary and Middle Schools 
Technical Assistance Center for a participating district. 

Welch, C. & Davis, C. (2001, October). EMSTAC and CECP: Promoting culturally competent 

service delivery to reduce disproportionality for diverse learners with behavior disorders. 
Presentation conducted at the Council for Children with Behavioral Disorders Annual 
Conference, Atlanta, GA. 

Welch, C. & Davis, C. (2002, April). Comrise Project. Panel discussion conducted at the Council for 
Exceptional Children Annual Convention and Exposition, New York, NY. 

Welch, C., Mead, J. & Sims, A. (2001, Febmary). Challenges and strategies for disproportionality: 
Procedures, processes, and promising practices. Presentation conducted at the U.S. 
Department of Education, Office of Special Education Programs (OSEP) Personnel 
Preparation Conference, Washington, DC. 

Welch, C., Ruedel, K. & Sims, A. (2000, December). Training on the process of change and 

disproportionate representation of minority youth in special education. Training conducted 
on behalf of the Elementary and Middle Schools Technical Assistance Center for a 
participating district. 

Welch, C. & Osher, D(April, 2002). Culturally responsive strategies for reducing 

disproportionality: Utilizing partnerships to create equitable learning communities. Poster 
presented at the Council for Exceptional Children Annual Conference and Exposition, New 
York, NY. 

Wood-Gamett, S., Welch, C. &Sims, A. (2000, November). Commitment, cooperation and 

collaboration: Federally funded projects and issues of disproportionate representation in 
special education. Presentation conducted at the National Alliance of Black School Educators 
(NABSE) Annual Conference, Philadelphia, PA. 

Wood-Gamett, S., Welch, C. & Timmes, J. (2000, November). Commitment, cooperation, and 

collaboration: Strategies for addressing issues related to African American youth in special 
education. Presentation conducted at the National Alliance of Black School Educators 
Annual Conference, Philadelphia, PA. 

Woodmff, D. (1999, April). School-wide discipline and behavioral assessment: Ensuring successful 
school experiences for all students. Poster presented at the Council for Exceptional Children 
Annual Conference and Exposition, Charlotte, NC. 
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Woodruff, D. (2000, April). Improving student outcomes: The Elementary and Middle School 

Technical Assistance Center. Presentation conducted at the Council of Chief State School 
Officers Project to Improve Achievement in High Poverty Schools, Long Beach, CA. 

Woodruff, D. (2000, October). School-wide discipline: Ensuring successful school experiences for 
all students. Presentation conducted at the Council for Exceptional Children and the Division 
for Culturally and Linguistically Diverse Exceptional Learners 2000 Symposium, 
Albuquerque, NM. 

Woodruff, D. (2000, October). Using technical assistance in schools to improve services for children 
with and at risk for disabilities. Presentation conducted at the Montana Conference of 
Education Leadership, Billings, MT. 

Woodruff, D. (2000, November). Effective partnerships with schools to improve services for children 
with disabilities. Presentation conducted at the International Adolescent Conference, 

Portland, OR. 

Woodruff, D. (2001, June). Creating a school environment that promotes mental health and 

addresses the emotional and behavioral needs of all students: A three tiered approach. 
Presentation conduction at the IDEA Partnerships National Summit, Washington, DC. 

Woodruff, D., Ball, W. & Paroz, J. (2001, March). Effective practices: The Elementary and Middle 
Schools Technical Assistance Center. Presentation conducted for a participating district. 

Woodruff, D., Ball, W. & Paroz, J. (2001, November). Improving student outcomes through effective 
partnerships: Lessons learned. Presentation conducted at the Council of Administrators of 
Special Education, 12* Annual CASE Conference, Irvine, CA. 

Woodruff, D., Jackson, S., & Shami, M. (2002, April). Choosing Effective Behavioral Supports for 
Students with Disabilities. Presentation conducted at the Council for Exceptional Children 
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